2024-04-23T21:11:03.629 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-23T21:11:03.629 DEBUG:teuthology.run:Teuthology command: teuthology --description rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_latest} workloads/rbd-mirror-journal-stress-workunit} --archive /home/teuthworker/archive/teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi/7670210 --name teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi --verbose --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi/7670210/orig.config.yaml 2024-04-23T21:11:03.676 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T21:11:03.759 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi/7670210 branch: main description: rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_latest} workloads/rbd-mirror-journal-stress-workunit} email: ceph-qa@ceph.com first_in_suite: false job_id: '7670210' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: run two ceph clusters and install rbd-mirror - desc: configure the permissions for client.mirror - desc: 2 ceph clusters with 1 mon and 3 osds each - desc: run four rbd-mirror daemons per cluster - desc: run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon name: teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: debug rbd: 20 debug rbd_mirror: 15 log to stderr: false rbd default features: 125 rbd mirror image policy type: simple client.mirror.0: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.1: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.2: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.3: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME sha1: 8254cb5a247caa8e3d35e534a1fefe2362a0a4b8 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 8254cb5a247caa8e3d35e534a1fefe2362a0a4b8 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f owner: scheduled_teuthology@teuthology priority: 950 repo: https://git.ceph.com/ceph.git roles: - - cluster1.mon.a - cluster1.mgr.x - cluster2.mgr.x - cluster1.osd.0 - cluster1.osd.1 - cluster1.osd.2 - cluster1.client.0 - cluster2.client.0 - - cluster2.mon.a - cluster2.osd.0 - cluster2.osd.1 - cluster2.osd.2 - cluster1.client.mirror - cluster1.client.mirror.0 - cluster1.client.mirror.1 - cluster1.client.mirror.2 - cluster1.client.mirror.3 - cluster1.client.mirror.4 - cluster1.client.mirror.5 - cluster1.client.mirror.6 - cluster2.client.mirror - cluster2.client.mirror.0 - cluster2.client.mirror.1 - cluster2.client.mirror.2 - cluster2.client.mirror.3 - cluster2.client.mirror.4 - cluster2.client.mirror.5 - cluster2.client.mirror.6 seed: 4807 sha1: 8254cb5a247caa8e3d35e534a1fefe2362a0a4b8 sleep_before_teardown: 0 subset: 19/128 suite: rbd suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f targets: smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMX5JWB1fO38Pml0tJfiB3TVJ7QYRvuufJe0BcdYHxgyd5yTnCGYAITfnpWXKHm3rywGVVbfw5bTG0hTY1OpztA= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBS9ZCMITkD7l57bj+JwW3X3sW3qR6Mz57CYuRA4xv0lgLTn9nZ1ZGAs6/+IyWptTbmmMLbQ/Re0zEwOe4t0Brk= tasks: - install: extra_packages: - rbd-mirror - ceph: cluster: cluster1 - ceph: cluster: cluster2 - exec: cluster1.client.mirror.0: - sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd' - rbd-mirror: client: cluster1.client.mirror.0 thrash: true - rbd-mirror: client: cluster1.client.mirror.1 thrash: true - rbd-mirror: client: cluster1.client.mirror.2 thrash: true - rbd-mirror: client: cluster1.client.mirror.3 thrash: true - rbd-mirror: client: cluster2.client.mirror.0 thrash: true - rbd-mirror: client: cluster2.client.mirror.1 thrash: true - rbd-mirror: client: cluster2.client.mirror.2 thrash: true - rbd-mirror: client: cluster2.client.mirror.3 thrash: true - rbd-mirror-thrash: cluster: cluster1 - rbd-mirror-thrash: cluster: cluster2 - workunit: clients: cluster1.client.mirror: - rbd/rbd_mirror_stress.sh env: CEPH_ARGS: '' MIRROR_POOL_MODE: pool RBD_MIRROR_INSTANCES: '4' RBD_MIRROR_MODE: journal RBD_MIRROR_USE_EXISTING_CLUSTER: '1' RBD_MIRROR_USE_RBD_MIRROR: '1' timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-23_20:16:13 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-23T21:11:03.760 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/qa; will attempt to use it 2024-04-23T21:11:03.760 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/qa/tasks 2024-04-23T21:11:03.760 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-23T21:11:03.762 INFO:teuthology.task.internal:Checking packages... 2024-04-23T21:11:03.796 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8254cb5a247caa8e3d35e534a1fefe2362a0a4b8' 2024-04-23T21:11:03.796 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-23T21:11:03.796 INFO:teuthology.packaging:ref: None 2024-04-23T21:11:03.797 INFO:teuthology.packaging:tag: None 2024-04-23T21:11:03.797 INFO:teuthology.packaging:branch: main 2024-04-23T21:11:03.797 INFO:teuthology.packaging:sha1: 8254cb5a247caa8e3d35e534a1fefe2362a0a4b8 2024-04-23T21:11:03.797 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-04-23T21:11:04.182 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3212.g8254cb5a 2024-04-23T21:11:04.183 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-23T21:11:04.190 INFO:teuthology.task.internal:no buildpackages task found 2024-04-23T21:11:04.190 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-23T21:11:04.196 INFO:teuthology.task.internal:Saving configuration 2024-04-23T21:11:04.213 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-23T21:11:04.219 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T21:11:04.247 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi/7670210', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 21:05:46.458245', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMX5JWB1fO38Pml0tJfiB3TVJ7QYRvuufJe0BcdYHxgyd5yTnCGYAITfnpWXKHm3rywGVVbfw5bTG0hTY1OpztA='} 2024-04-23T21:11:04.267 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi/7670210', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 21:05:46.456547', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBS9ZCMITkD7l57bj+JwW3X3sW3qR6Mz57CYuRA4xv0lgLTn9nZ1ZGAs6/+IyWptTbmmMLbQ/Re0zEwOe4t0Brk='} 2024-04-23T21:11:04.267 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-23T21:11:04.273 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'] 2024-04-23T21:11:04.273 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6'] 2024-04-23T21:11:04.273 INFO:teuthology.run_tasks:Running task console_log... 2024-04-23T21:11:04.345 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa658c0d040>, signals=[15]) 2024-04-23T21:11:04.345 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-23T21:11:04.352 INFO:teuthology.task.internal:Opening connections... 2024-04-23T21:11:04.354 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-04-23T21:11:04.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T21:11:04.437 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-04-23T21:11:04.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T21:11:04.511 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-23T21:11:04.518 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-04-23T21:11:04.535 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-04-23T21:11:04.536 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-04-23T21:11:04.592 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-04-23T21:11:04.592 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-04-23T21:11:04.593 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-04-23T21:11:04.593 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-04-23T21:11:04.593 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-04-23T21:11:04.593 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-04-23T21:11:04.593 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T21:11:04.593 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-04-23T21:11:04.594 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-04-23T21:11:04.594 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T21:11:04.594 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-04-23T21:11:04.594 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T21:11:04.594 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T21:11:04.594 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T21:11:04.595 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-04-23T21:11:04.616 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-04-23T21:11:04.634 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-04-23T21:11:04.635 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-04-23T21:11:04.693 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-04-23T21:11:04.694 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-04-23T21:11:04.694 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-04-23T21:11:04.694 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-04-23T21:11:04.694 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-04-23T21:11:04.694 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-04-23T21:11:04.694 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T21:11:04.694 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-04-23T21:11:04.694 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-04-23T21:11:04.694 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T21:11:04.694 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-04-23T21:11:04.694 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T21:11:04.694 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T21:11:04.694 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T21:11:04.695 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-04-23T21:11:04.715 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-23T21:11:04.725 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-23T21:11:04.731 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-23T21:11:04.731 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-04-23T21:11:04.733 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-04-23T21:11:04.751 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-23T21:11:04.756 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-23T21:11:04.757 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-04-23T21:11:04.793 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-04-23T21:11:04.896 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-23T21:11:04.956 INFO:teuthology.run_tasks:Running task kernel... 2024-04-23T21:11:04.970 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-23T21:11:04.971 INFO:teuthology.task.kernel:config {'cluster1.mon.a': {'kdb': 1, 'sha1': 'distro'}, 'cluster2.mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-23T21:11:04.971 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T21:11:04.971 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T21:11:04.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T21:11:04.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T21:11:04.989 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-04-23T21:11:04.989 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-04-23T21:11:05.047 INFO:teuthology.orchestra.run.smithi150.stdout:5.14.0-437.el9.x86_64 2024-04-23T21:11:05.047 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-437.el9.x86_64 2024-04-23T21:11:05.048 INFO:teuthology.task.kernel:Running kernel on smithi150: 5.14.0-437.el9.x86_64 2024-04-23T21:11:05.048 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum install -y kernel 2024-04-23T21:11:05.048 INFO:teuthology.task.kernel:Running kernel on smithi148: 5.14.0-437.el9.x86_64 2024-04-23T21:11:05.048 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum install -y kernel 2024-04-23T21:11:07.393 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-23T21:11:07.952 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 7.1 MB/s | 8.0 MB 00:01 2024-04-23T21:11:09.585 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-23T21:11:09.870 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-23T21:11:13.611 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-23T21:11:13.912 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-04-23T21:11:15.563 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-04-23T21:11:15.679 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 16 kB 00:00 2024-04-23T21:11:17.066 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-23T21:11:22.154 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-23T21:11:23.986 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-23T21:11:23.986 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-23T21:11:24.073 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:11:24.074 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-04-23T21:11:24.074 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:11:24.185 DEBUG:teuthology.orchestra.run.smithi148:> echo no | sudo yum reinstall kernel || true 2024-04-23T21:11:24.685 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:02 ago on Tue 23 Apr 2024 09:11:22 PM UTC. 2024-04-23T21:11:24.806 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:11:24.816 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:11:24.817 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-04-23T21:11:24.817 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:11:24.817 INFO:teuthology.orchestra.run.smithi148.stdout:Reinstalling: 2024-04-23T21:11:24.817 INFO:teuthology.orchestra.run.smithi148.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T21:11:24.817 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:11:24.817 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:11:24.817 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:11:24.817 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:11:24.817 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 255 k 2024-04-23T21:11:24.817 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 0 2024-04-23T21:11:24.817 INFO:teuthology.orchestra.run.smithi148.stderr:Operation aborted. 2024-04-23T21:11:24.875 INFO:teuthology.orchestra.run.smithi148.stdout:Is this ok [y/N]: 2024-04-23T21:11:24.876 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum reinstall -y kernel || true 2024-04-23T21:11:25.370 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:03 ago on Tue 23 Apr 2024 09:11:22 PM UTC. 2024-04-23T21:11:25.485 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:11:25.486 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:11:25.486 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-04-23T21:11:25.486 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:11:25.487 INFO:teuthology.orchestra.run.smithi148.stdout:Reinstalling: 2024-04-23T21:11:25.487 INFO:teuthology.orchestra.run.smithi148.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T21:11:25.487 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:11:25.487 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:11:25.487 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:11:25.487 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:11:25.487 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 255 k 2024-04-23T21:11:25.487 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 0 2024-04-23T21:11:25.487 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:11:26.740 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-437.el9.x86_64.rpm 261 kB/s | 255 kB 00:00 2024-04-23T21:11:26.740 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:11:26.741 INFO:teuthology.orchestra.run.smithi148.stdout:Total 203 kB/s | 255 kB 00:01 2024-04-23T21:11:26.762 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:11:26.947 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:11:26.947 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:11:26.952 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:11:26.952 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:11:27.089 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:11:27.166 INFO:teuthology.orchestra.run.smithi148.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T21:11:27.166 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T21:11:27.794 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T21:11:27.795 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T21:11:28.440 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T21:11:28.440 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:11:28.440 INFO:teuthology.orchestra.run.smithi148.stdout:Reinstalled: 2024-04-23T21:11:28.440 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-23T21:11:28.440 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:11:28.441 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:11:28.628 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T21:11:28.675 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-23T21:11:28.675 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-23T21:11:28.675 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-23T21:11:28.675 INFO:teuthology.task.kernel:Enabling kdb on cluster1.mon.a... 2024-04-23T21:11:28.675 DEBUG:teuthology.orchestra.run.smithi148:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-23T21:11:28.707 INFO:teuthology.orchestra.run.smithi148.stdout:ttyS1 2024-04-23T21:11:28.735 DEBUG:teuthology.parallel:result is None 2024-04-23T21:11:34.724 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-23T21:11:39.828 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-23T21:11:41.710 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-23T21:11:41.711 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-23T21:11:41.813 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:11:41.814 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-04-23T21:11:41.814 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:11:41.912 DEBUG:teuthology.orchestra.run.smithi150:> echo no | sudo yum reinstall kernel || true 2024-04-23T21:11:42.414 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:03 ago on Tue 23 Apr 2024 09:11:39 PM UTC. 2024-04-23T21:11:42.543 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:11:42.549 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:11:42.549 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-04-23T21:11:42.549 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:11:42.549 INFO:teuthology.orchestra.run.smithi150.stdout:Reinstalling: 2024-04-23T21:11:42.549 INFO:teuthology.orchestra.run.smithi150.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T21:11:42.549 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:11:42.549 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:11:42.550 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:11:42.550 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:11:42.550 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 255 k 2024-04-23T21:11:42.550 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 0 2024-04-23T21:11:42.550 INFO:teuthology.orchestra.run.smithi150.stderr:Operation aborted. 2024-04-23T21:11:42.608 INFO:teuthology.orchestra.run.smithi150.stdout:Is this ok [y/N]: 2024-04-23T21:11:42.608 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum reinstall -y kernel || true 2024-04-23T21:11:43.103 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:04 ago on Tue 23 Apr 2024 09:11:39 PM UTC. 2024-04-23T21:11:43.219 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:11:43.219 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-04-23T21:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:Reinstalling: 2024-04-23T21:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T21:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 255 k 2024-04-23T21:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 0 2024-04-23T21:11:43.220 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:11:44.006 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-5.14.0-437.el9.x86_64.rpm 393 kB/s | 255 kB 00:00 2024-04-23T21:11:44.006 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:11:44.007 INFO:teuthology.orchestra.run.smithi150.stdout:Total 324 kB/s | 255 kB 00:00 2024-04-23T21:11:44.022 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:11:44.208 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:11:44.208 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:11:44.213 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:11:44.213 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:11:44.341 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:11:44.415 INFO:teuthology.orchestra.run.smithi150.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T21:11:44.416 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T21:11:45.150 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T21:11:45.151 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T21:11:45.840 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T21:11:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:11:45.841 INFO:teuthology.orchestra.run.smithi150.stdout:Reinstalled: 2024-04-23T21:11:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-23T21:11:45.841 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:11:45.841 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:11:46.009 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T21:11:46.056 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-23T21:11:46.056 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-23T21:11:46.056 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-23T21:11:46.057 INFO:teuthology.task.kernel:Enabling kdb on cluster2.mon.a... 2024-04-23T21:11:46.057 DEBUG:teuthology.orchestra.run.smithi150:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-23T21:11:46.094 INFO:teuthology.orchestra.run.smithi150.stdout:ttyS1 2024-04-23T21:11:46.111 DEBUG:teuthology.parallel:result is None 2024-04-23T21:11:46.111 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-23T21:11:46.118 INFO:teuthology.task.internal:Creating test directory... 2024-04-23T21:11:46.118 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T21:11:46.121 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T21:11:46.165 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-23T21:11:46.172 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-23T21:11:46.181 INFO:teuthology.task.internal:Creating archive directory... 2024-04-23T21:11:46.181 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T21:11:46.184 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T21:11:46.238 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-23T21:11:46.245 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-23T21:11:46.245 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T21:11:46.257 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T21:11:46.289 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T21:11:46.299 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T21:11:46.305 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T21:11:46.314 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T21:11:46.316 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-23T21:11:46.323 INFO:teuthology.task.internal:Configuring sudo... 2024-04-23T21:11:46.323 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T21:11:46.345 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T21:11:46.379 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-23T21:11:46.389 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-23T21:11:46.389 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T21:11:46.411 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T21:11:46.434 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T21:11:46.506 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T21:11:46.556 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T21:11:46.640 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T21:11:46.679 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:11:46.679 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T21:11:46.744 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T21:11:46.782 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T21:11:46.828 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T21:11:46.910 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T21:11:46.948 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:11:46.949 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T21:11:47.012 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-04-23T21:11:47.014 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-04-23T21:11:47.054 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T21:11:47.090 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T21:11:47.551 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-23T21:11:47.559 INFO:teuthology.task.internal:Starting timer... 2024-04-23T21:11:47.559 INFO:teuthology.run_tasks:Running task pcp... 2024-04-23T21:11:47.571 INFO:teuthology.run_tasks:Running task selinux... 2024-04-23T21:11:47.580 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-23T21:11:47.580 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-04-23T21:11:47.649 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-04-23T21:11:47.651 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-04-23T21:11:47.715 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-04-23T21:11:47.716 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-23T21:11:47.716 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-04-23T21:11:47.736 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-04-23T21:11:47.737 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-04-23T21:11:47.754 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-04-23T21:11:47.798 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi148.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive'} 2024-04-23T21:11:47.798 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T21:11:47.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T21:11:47.825 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T21:11:47.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T21:11:47.849 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-23T21:11:47.849 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-04-23T21:11:47.897 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-04-23T21:11:47.921 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-23T21:11:47.932 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-23T21:11:47.948 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-23T21:11:47.949 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi148.front.sepia.ceph.com,smithi150.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-23T21:15:18.629 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi148.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com')] 2024-04-23T21:15:18.631 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-04-23T21:15:18.632 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T21:15:18.710 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-04-23T21:15:18.793 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-04-23T21:15:18.793 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-23T21:15:18.793 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T21:15:18.865 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-04-23T21:15:18.945 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-23T21:15:18.945 INFO:teuthology.run_tasks:Running task clock... 2024-04-23T21:15:18.956 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-23T21:15:18.956 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T21:15:18.957 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T21:15:18.959 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T21:15:18.959 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T21:15:18.986 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T21:15:18.999 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T21:15:19.013 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T21:15:19.021 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-04-23T21:15:19.027 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T21:15:19.031 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-04-23T21:15:19.044 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T21:15:19.049 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-04-23T21:15:19.057 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T21:15:19.059 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-04-23T21:15:19.072 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T21:15:19.085 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T21:15:19.109 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-04-23T21:15:19.113 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T21:15:19.113 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-04-23T21:15:19.113 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T21:15:19.113 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T21:15:19.113 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T21:15:19.113 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T21:15:19.140 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-04-23T21:15:19.144 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T21:15:19.144 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-04-23T21:15:19.144 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T21:15:19.145 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T21:15:19.145 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T21:15:19.145 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T21:15:19.145 INFO:teuthology.run_tasks:Running task install... 2024-04-23T21:15:19.156 DEBUG:teuthology.task.install:project ceph 2024-04-23T21:15:19.156 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8254cb5a247caa8e3d35e534a1fefe2362a0a4b8'}} 2024-04-23T21:15:19.157 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-mirror'], 'flavor': 'default', 'sha1': '8254cb5a247caa8e3d35e534a1fefe2362a0a4b8'} 2024-04-23T21:15:19.157 INFO:teuthology.task.install:Using flavor: default 2024-04-23T21:15:19.164 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-23T21:15:19.164 INFO:teuthology.task.install:extra packages: ['rbd-mirror'] 2024-04-23T21:15:19.165 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-mirror'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8254cb5a247caa8e3d35e534a1fefe2362a0a4b8', 'tag': None, 'wait_for_package': False} 2024-04-23T21:15:19.165 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8254cb5a247caa8e3d35e534a1fefe2362a0a4b8 2024-04-23T21:15:19.168 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-mirror'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8254cb5a247caa8e3d35e534a1fefe2362a0a4b8', 'tag': None, 'wait_for_package': False} 2024-04-23T21:15:19.168 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8254cb5a247caa8e3d35e534a1fefe2362a0a4b8 2024-04-23T21:15:19.313 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/8254cb5a247caa8e3d35e534a1fefe2362a0a4b8/centos/9/flavors/default/ 2024-04-23T21:15:19.314 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3212.g8254cb5a 2024-04-23T21:15:19.414 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/8254cb5a247caa8e3d35e534a1fefe2362a0a4b8/centos/9/flavors/default/ 2024-04-23T21:15:19.414 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3212.g8254cb5a 2024-04-23T21:15:19.431 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/8254cb5a247caa8e3d35e534a1fefe2362a0a4b8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/8254cb5a247caa8e3d35e534a1fefe2362a0a4b8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/8254cb5a247caa8e3d35e534a1fefe2362a0a4b8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T21:15:19.431 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:15:19.431 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T21:15:19.456 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-mirror on remote rpm x86_64 2024-04-23T21:15:19.456 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8254cb5a247caa8e3d35e534a1fefe2362a0a4b8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T21:15:19.520 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T21:15:19.593 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T21:15:19.619 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/8254cb5a247caa8e3d35e534a1fefe2362a0a4b8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/8254cb5a247caa8e3d35e534a1fefe2362a0a4b8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/8254cb5a247caa8e3d35e534a1fefe2362a0a4b8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T21:15:19.619 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:15:19.619 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T21:15:19.645 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-mirror on remote rpm x86_64 2024-04-23T21:15:19.645 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8254cb5a247caa8e3d35e534a1fefe2362a0a4b8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T21:15:19.656 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-04-23T21:15:19.658 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-04-23T21:15:19.711 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T21:15:19.785 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T21:15:19.848 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-04-23T21:15:19.850 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-04-23T21:15:19.883 INFO:teuthology.orchestra.run.smithi150.stdout:48 files removed 2024-04-23T21:15:19.911 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw 2024-04-23T21:15:20.072 INFO:teuthology.orchestra.run.smithi148.stdout:48 files removed 2024-04-23T21:15:20.099 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw 2024-04-23T21:15:20.865 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 164 kB/s | 83 kB 00:00 2024-04-23T21:15:20.932 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 109 kB/s | 83 kB 00:00 2024-04-23T21:15:21.349 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-23T21:15:21.416 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-23T21:15:21.683 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 6.4 kB/s | 1.9 kB 00:00 2024-04-23T21:15:22.084 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-04-23T21:15:26.124 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 8.0 MB 00:04 2024-04-23T21:15:29.333 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 8.9 MB/s | 19 MB 00:02 2024-04-23T21:15:31.525 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 876 kB/s | 8.0 MB 00:09 2024-04-23T21:15:33.709 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.2 MB 00:00 2024-04-23T21:15:35.634 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 6.3 MB/s | 19 MB 00:03 2024-04-23T21:15:36.420 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 10 kB/s | 16 kB 00:01 2024-04-23T21:15:38.281 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-23T21:15:40.393 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.2 MB 00:01 2024-04-23T21:15:42.779 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-23T21:15:43.478 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 9.1 kB/s | 1.7 kB 00:00 2024-04-23T21:15:45.479 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 8.1 MB/s | 21 MB 00:02 2024-04-23T21:15:45.604 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:15:45.607 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:15:45.607 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-04-23T21:15:45.607 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:15:45.607 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T21:15:45.607 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 11 M 2024-04-23T21:15:45.607 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-04-23T21:15:45.607 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 3.4 M 2024-04-23T21:15:45.607 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 3.1 M 2024-04-23T21:15:45.607 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-04-23T21:15:45.607 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T21:15:45.607 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 5.4 M 2024-04-23T21:15:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 21 M 2024-04-23T21:15:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 25 k 2024-04-23T21:15:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T21:15:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T21:15:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T21:15:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 747 k 2024-04-23T21:15:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T21:15:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T21:15:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T21:15:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 500 k 2024-04-23T21:15:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T21:15:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 5.4 M 2024-04-23T21:15:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T21:15:45.608 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T21:15:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T21:15:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 45 k 2024-04-23T21:15:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 135 k 2024-04-23T21:15:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 162 k 2024-04-23T21:15:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T21:15:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 320 k 2024-04-23T21:15:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 298 k 2024-04-23T21:15:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 100 k 2024-04-23T21:15:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T21:15:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T21:15:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T21:15:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T21:15:45.609 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-04-23T21:15:45.609 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T21:15:45.610 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:15:45.610 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:15:45.610 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:15:45.610 INFO:teuthology.orchestra.run.smithi148.stdout:Install 30 Packages 2024-04-23T21:15:45.610 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 2 Packages 2024-04-23T21:15:45.610 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:15:45.610 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 60 M 2024-04-23T21:15:45.610 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:15:47.271 INFO:teuthology.orchestra.run.smithi148.stdout:(1/32): ceph-radosgw-19.0.0-3212.g8254cb5a.el9. 8.1 MB/s | 11 MB 00:01 2024-04-23T21:15:47.321 INFO:teuthology.orchestra.run.smithi148.stdout:(2/32): ceph-selinux-19.0.0-3212.g8254cb5a.el9. 506 kB/s | 25 kB 00:00 2024-04-23T21:15:47.438 INFO:teuthology.orchestra.run.smithi148.stdout:(3/32): libcephfs2-19.0.0-3212.g8254cb5a.el9.x8 6.3 MB/s | 747 kB 00:00 2024-04-23T21:15:47.538 INFO:teuthology.orchestra.run.smithi148.stdout:(4/32): ceph-base-19.0.0-3212.g8254cb5a.el9.x86 3.4 MB/s | 5.4 MB 00:01 2024-04-23T21:15:47.564 INFO:teuthology.orchestra.run.smithi148.stdout:(5/32): libradosstriper1-19.0.0-3212.g8254cb5a. 3.9 MB/s | 500 kB 00:00 2024-04-23T21:15:47.630 INFO:teuthology.orchestra.run.smithi148.stdout:(6/32): python3-ceph-argparse-19.0.0-3212.g8254 675 kB/s | 45 kB 00:00 2024-04-23T21:15:47.947 INFO:teuthology.orchestra.run.smithi148.stdout:(7/32): python3-ceph-common-19.0.0-3212.g8254cb 425 kB/s | 135 kB 00:00 2024-04-23T21:15:48.197 INFO:teuthology.orchestra.run.smithi148.stdout:(8/32): ceph-common-19.0.0-3212.g8254cb5a.el9.x 9.5 MB/s | 21 MB 00:02 2024-04-23T21:15:48.223 INFO:teuthology.orchestra.run.smithi148.stdout:(9/32): python3-cephfs-19.0.0-3212.g8254cb5a.el 589 kB/s | 162 kB 00:00 2024-04-23T21:15:48.248 INFO:teuthology.orchestra.run.smithi148.stdout:(10/32): python3-rados-19.0.0-3212.g8254cb5a.el 6.3 MB/s | 320 kB 00:00 2024-04-23T21:15:48.332 INFO:teuthology.orchestra.run.smithi148.stdout:(11/32): librgw2-19.0.0-3212.g8254cb5a.el9.x86_ 6.8 MB/s | 5.4 MB 00:00 2024-04-23T21:15:48.357 INFO:teuthology.orchestra.run.smithi148.stdout:(12/32): python3-rbd-19.0.0-3212.g8254cb5a.el9. 2.2 MB/s | 298 kB 00:00 2024-04-23T21:15:48.382 INFO:teuthology.orchestra.run.smithi148.stdout:(13/32): python3-rgw-19.0.0-3212.g8254cb5a.el9. 746 kB/s | 100 kB 00:00 2024-04-23T21:15:48.532 INFO:teuthology.orchestra.run.smithi148.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 301 kB/s | 45 kB 00:00 2024-04-23T21:15:48.557 INFO:teuthology.orchestra.run.smithi148.stdout:(15/32): boost-program-options-1.75.0-8.el9.x86 474 kB/s | 107 kB 00:00 2024-04-23T21:15:48.583 INFO:teuthology.orchestra.run.smithi148.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 709 kB/s | 160 kB 00:00 2024-04-23T21:15:48.642 INFO:teuthology.orchestra.run.smithi148.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 717 kB/s | 42 kB 00:00 2024-04-23T21:15:48.667 INFO:teuthology.orchestra.run.smithi148.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-04-23T21:15:48.700 INFO:teuthology.orchestra.run.smithi148.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-04-23T21:15:48.717 INFO:teuthology.orchestra.run.smithi148.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 716 kB/s | 36 kB 00:00 2024-04-23T21:15:48.743 INFO:teuthology.orchestra.run.smithi148.stdout:(21/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-04-23T21:15:48.760 INFO:teuthology.orchestra.run.smithi148.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 805 kB/s | 47 kB 00:00 2024-04-23T21:15:49.152 INFO:teuthology.orchestra.run.smithi148.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 66 kB/s | 26 kB 00:00 2024-04-23T21:15:49.327 INFO:teuthology.orchestra.run.smithi148.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 278 kB/s | 49 kB 00:00 2024-04-23T21:15:49.352 INFO:teuthology.orchestra.run.smithi148.stdout:(25/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 487 kB/s | 309 kB 00:00 2024-04-23T21:15:49.486 INFO:teuthology.orchestra.run.smithi148.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 425 kB/s | 67 kB 00:00 2024-04-23T21:15:49.611 INFO:teuthology.orchestra.run.smithi148.stdout:(27/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 838 kB 00:00 2024-04-23T21:15:49.703 INFO:teuthology.orchestra.run.smithi148.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 884 kB/s | 192 kB 00:00 2024-04-23T21:15:49.779 INFO:teuthology.orchestra.run.smithi148.stdout:(29/32): libarrow-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 4.4 MB 00:01 2024-04-23T21:15:49.820 INFO:teuthology.orchestra.run.smithi148.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-23T21:15:50.062 INFO:teuthology.orchestra.run.smithi148.stdout:(31/32): librbd1-19.0.0-3212.g8254cb5a.el9.x86_ 11 MB/s | 3.1 MB 00:00 2024-04-23T21:15:50.479 INFO:teuthology.orchestra.run.smithi148.stdout:(32/32): librados2-19.0.0-3212.g8254cb5a.el9.x8 4.4 MB/s | 3.4 MB 00:00 2024-04-23T21:15:50.485 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:15:50.485 INFO:teuthology.orchestra.run.smithi148.stdout:Total 12 MB/s | 60 MB 00:04 2024-04-23T21:15:50.570 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-23T21:15:50.583 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:15:50.620 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:15:50.620 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:15:51.089 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:15:51.090 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:15:52.568 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:15:52.654 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-23T21:15:52.684 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-04-23T21:15:52.709 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-04-23T21:15:52.732 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-23T21:15:52.754 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:15:52.754 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-23T21:15:52.756 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:15:52.756 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-23T21:15:52.756 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:15:52.757 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-23T21:15:52.757 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 11 M 2024-04-23T21:15:52.757 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-04-23T21:15:52.757 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 3.4 M 2024-04-23T21:15:52.757 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 3.1 M 2024-04-23T21:15:52.757 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-04-23T21:15:52.757 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T21:15:52.757 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 5.4 M 2024-04-23T21:15:52.757 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 21 M 2024-04-23T21:15:52.757 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 25 k 2024-04-23T21:15:52.757 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T21:15:52.757 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T21:15:52.757 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T21:15:52.757 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 747 k 2024-04-23T21:15:52.758 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T21:15:52.758 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T21:15:52.758 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T21:15:52.758 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 500 k 2024-04-23T21:15:52.758 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T21:15:52.758 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 5.4 M 2024-04-23T21:15:52.758 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T21:15:52.758 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T21:15:52.758 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T21:15:52.758 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 45 k 2024-04-23T21:15:52.758 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 135 k 2024-04-23T21:15:52.758 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 162 k 2024-04-23T21:15:52.758 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T21:15:52.759 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 320 k 2024-04-23T21:15:52.759 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 298 k 2024-04-23T21:15:52.759 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 100 k 2024-04-23T21:15:52.759 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T21:15:52.759 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T21:15:52.759 INFO:teuthology.orchestra.run.smithi150.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T21:15:52.759 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T21:15:52.759 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-04-23T21:15:52.759 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T21:15:52.759 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:15:52.759 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:15:52.759 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:15:52.759 INFO:teuthology.orchestra.run.smithi150.stdout:Install 30 Packages 2024-04-23T21:15:52.759 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 2 Packages 2024-04-23T21:15:52.759 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:15:52.760 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 60 M 2024-04-23T21:15:52.760 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:15:52.957 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-23T21:15:53.011 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-23T21:15:53.392 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-04-23T21:15:53.452 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:19.0.0-3212.g8254cb5a.el9.x86_64 7/34 2024-04-23T21:15:53.527 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:19.0.0-3212.g8254cb5a.el9.x86_64 7/34 2024-04-23T21:15:53.617 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:19.0.0-3212.g8254cb5a.el9.x86_64 8/34 2024-04-23T21:15:53.654 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:19.0.0-3212.g8254cb5a.el9.x86_64 9/34 2024-04-23T21:15:53.700 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:19.0.0-3212.g8254cb5a.el9.x86_64 9/34 2024-04-23T21:15:53.729 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-04-23T21:15:53.772 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:19.0.0-3212.g8254cb5a.el9. 11/34 2024-04-23T21:15:53.842 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:19.0.0-3212.g8254cb5a.el9.x86_64 12/34 2024-04-23T21:15:53.864 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:19.0.0-3212.g8254cb5a.el9.x86_6 13/34 2024-04-23T21:15:53.928 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3212.g8254cb5a.el9.x86_6 13/34 2024-04-23T21:15:53.963 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-04-23T21:15:54.089 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-04-23T21:15:54.121 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-04-23T21:15:54.155 INFO:teuthology.orchestra.run.smithi150.stdout:(1/32): ceph-base-19.0.0-3212.g8254cb5a.el9.x86 5.9 MB/s | 5.4 MB 00:00 2024-04-23T21:15:54.161 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-04-23T21:15:54.187 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-04-23T21:15:54.205 INFO:teuthology.orchestra.run.smithi150.stdout:(2/32): ceph-selinux-19.0.0-3212.g8254cb5a.el9. 507 kB/s | 25 kB 00:00 2024-04-23T21:15:54.216 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-04-23T21:15:54.249 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-04-23T21:15:54.305 INFO:teuthology.orchestra.run.smithi150.stdout:(3/32): libcephfs2-19.0.0-3212.g8254cb5a.el9.x8 7.3 MB/s | 747 kB 00:00 2024-04-23T21:15:54.364 INFO:teuthology.orchestra.run.smithi150.stdout:(4/32): libradosstriper1-19.0.0-3212.g8254cb5a. 8.4 MB/s | 500 kB 00:00 2024-04-23T21:15:54.564 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-04-23T21:15:54.605 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:19.0.0-3212.g8254cb5a.el9.x86_64 22/34 2024-04-23T21:15:54.694 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:19.0.0-3212.g8254cb5a.el9.x86_64 22/34 2024-04-23T21:15:54.718 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:19.0.0-3212.g8254cb5a.el9.x86_64 23/34 2024-04-23T21:15:54.754 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-23T21:15:54.756 INFO:teuthology.orchestra.run.smithi150.stdout:(5/32): librgw2-19.0.0-3212.g8254cb5a.el9.x86_6 14 MB/s | 5.4 MB 00:00 2024-04-23T21:15:54.823 INFO:teuthology.orchestra.run.smithi150.stdout:(6/32): python3-ceph-argparse-19.0.0-3212.g8254 680 kB/s | 45 kB 00:00 2024-04-23T21:15:54.948 INFO:teuthology.orchestra.run.smithi150.stdout:(7/32): python3-ceph-common-19.0.0-3212.g8254cb 1.1 MB/s | 135 kB 00:00 2024-04-23T21:15:55.007 INFO:teuthology.orchestra.run.smithi150.stdout:(8/32): python3-cephfs-19.0.0-3212.g8254cb5a.el 2.7 MB/s | 162 kB 00:00 2024-04-23T21:15:55.090 INFO:teuthology.orchestra.run.smithi150.stdout:(9/32): python3-rados-19.0.0-3212.g8254cb5a.el9 3.8 MB/s | 320 kB 00:00 2024-04-23T21:15:55.174 INFO:teuthology.orchestra.run.smithi150.stdout:(10/32): python3-rbd-19.0.0-3212.g8254cb5a.el9. 3.5 MB/s | 298 kB 00:00 2024-04-23T21:15:55.239 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-04-23T21:15:55.266 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:19.0.0-3212.g8254cb5a.el9.x86_64 26/34 2024-04-23T21:15:55.290 INFO:teuthology.orchestra.run.smithi150.stdout:(11/32): python3-rgw-19.0.0-3212.g8254cb5a.el9. 854 kB/s | 100 kB 00:00 2024-04-23T21:15:55.336 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:19.0.0-3212.g8254cb5a.el9.x86_64 26/34 2024-04-23T21:15:55.405 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:19.0.0-3212.g8254cb5a.el9.x86_64 27/34 2024-04-23T21:15:55.435 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:19.0.0-3212.g8254cb5a.el9.x8 28/34 2024-04-23T21:15:55.508 INFO:teuthology.orchestra.run.smithi150.stdout:(12/32): boost-program-options-1.75.0-8.el9.x86 493 kB/s | 107 kB 00:00 2024-04-23T21:15:55.616 INFO:teuthology.orchestra.run.smithi150.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-23T21:15:55.758 INFO:teuthology.orchestra.run.smithi150.stdout:(14/32): ceph-radosgw-19.0.0-3212.g8254cb5a.el9 4.2 MB/s | 11 MB 00:02 2024-04-23T21:15:55.783 INFO:teuthology.orchestra.run.smithi150.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 271 kB/s | 45 kB 00:00 2024-04-23T21:15:55.875 INFO:teuthology.orchestra.run.smithi150.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-04-23T21:15:55.942 INFO:teuthology.orchestra.run.smithi150.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 633 kB/s | 42 kB 00:00 2024-04-23T21:15:56.203 INFO:teuthology.orchestra.run.smithi150.stdout:(18/32): ceph-common-19.0.0-3212.g8254cb5a.el9. 7.2 MB/s | 21 MB 00:02 2024-04-23T21:15:56.234 INFO:teuthology.orchestra.run.smithi150.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 756 kB/s | 221 kB 00:00 2024-04-23T21:15:56.260 INFO:teuthology.orchestra.run.smithi150.stdout:(20/32): librdkafka-1.6.1-102.el9.x86_64.rpm 1.3 MB/s | 662 kB 00:00 2024-04-23T21:15:56.293 INFO:teuthology.orchestra.run.smithi150.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-04-23T21:15:56.318 INFO:teuthology.orchestra.run.smithi150.stdout:(22/32): qatlib-service-24.02.0-1.el9.x86_64.rp 311 kB/s | 36 kB 00:00 2024-04-23T21:15:56.536 INFO:teuthology.orchestra.run.smithi150.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 120 kB/s | 26 kB 00:00 2024-04-23T21:15:56.594 INFO:teuthology.orchestra.run.smithi150.stdout:(24/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 924 kB/s | 309 kB 00:00 2024-04-23T21:15:56.728 INFO:teuthology.orchestra.run.smithi150.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 505 kB/s | 67 kB 00:00 2024-04-23T21:15:56.745 INFO:teuthology.orchestra.run.smithi150.stdout:(26/32): liboath-2.6.7-2.el9.x86_64.rpm 232 kB/s | 49 kB 00:00 2024-04-23T21:15:57.012 INFO:teuthology.orchestra.run.smithi150.stdout:(27/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 838 kB 00:00 2024-04-23T21:15:57.279 INFO:teuthology.orchestra.run.smithi150.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 359 kB/s | 192 kB 00:00 2024-04-23T21:15:57.454 INFO:teuthology.orchestra.run.smithi150.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 3.6 MB/s | 1.6 MB 00:00 2024-04-23T21:15:57.688 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-3212.g8254cb5a.el9.x86_64 29/34 2024-04-23T21:15:57.781 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:19.0.0-3212.g8254cb5a.el9.x86_64 29/34 2024-04-23T21:15:57.971 INFO:teuthology.orchestra.run.smithi150.stdout:(30/32): librados2-19.0.0-3212.g8254cb5a.el9.x8 4.9 MB/s | 3.4 MB 00:00 2024-04-23T21:15:58.324 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-3212.g8254cb5a.el9.x86_64 29/34 2024-04-23T21:15:58.360 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:19.0.0-3212.g8254cb5a.el9.x86_64 30/34 2024-04-23T21:15:58.363 INFO:teuthology.orchestra.run.smithi150.stdout:(31/32): librbd1-19.0.0-3212.g8254cb5a.el9.x86_ 3.5 MB/s | 3.1 MB 00:00 2024-04-23T21:15:58.450 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:19.0.0-3212.g8254cb5a.el9.x86_64 30/34 2024-04-23T21:15:58.450 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T21:15:58.450 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T21:15:58.451 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:15:58.644 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:19.0.0-3212.g8254cb5a.el9.x86_64 31/34 2024-04-23T21:15:59.772 INFO:teuthology.orchestra.run.smithi150.stdout:(32/32): libarrow-9.0.0-10.el9.x86_64.rpm 1.3 MB/s | 4.4 MB 00:03 2024-04-23T21:15:59.774 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:15:59.775 INFO:teuthology.orchestra.run.smithi150.stdout:Total 8.6 MB/s | 60 MB 00:07 2024-04-23T21:15:59.879 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:15:59.915 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:15:59.916 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:16:00.384 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:16:00.385 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:16:01.852 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:16:01.930 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-23T21:16:01.968 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-04-23T21:16:01.993 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-04-23T21:16:02.016 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-23T21:16:02.038 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-23T21:16:02.233 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-23T21:16:02.287 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-23T21:16:02.609 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-04-23T21:16:02.729 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:19.0.0-3212.g8254cb5a.el9.x86_64 7/34 2024-04-23T21:16:02.811 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:19.0.0-3212.g8254cb5a.el9.x86_64 7/34 2024-04-23T21:16:02.898 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:19.0.0-3212.g8254cb5a.el9.x86_64 8/34 2024-04-23T21:16:02.938 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:19.0.0-3212.g8254cb5a.el9.x86_64 9/34 2024-04-23T21:16:02.984 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:19.0.0-3212.g8254cb5a.el9.x86_64 9/34 2024-04-23T21:16:03.013 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-04-23T21:16:03.056 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:19.0.0-3212.g8254cb5a.el9. 11/34 2024-04-23T21:16:03.124 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:19.0.0-3212.g8254cb5a.el9.x86_64 12/34 2024-04-23T21:16:03.157 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:19.0.0-3212.g8254cb5a.el9.x86_6 13/34 2024-04-23T21:16:03.221 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3212.g8254cb5a.el9.x86_6 13/34 2024-04-23T21:16:03.247 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-04-23T21:16:03.360 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-04-23T21:16:03.406 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-04-23T21:16:03.463 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-04-23T21:16:03.497 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-04-23T21:16:03.534 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-04-23T21:16:03.567 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-04-23T21:16:03.889 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-04-23T21:16:03.921 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:19.0.0-3212.g8254cb5a.el9.x86_64 22/34 2024-04-23T21:16:04.011 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:19.0.0-3212.g8254cb5a.el9.x86_64 22/34 2024-04-23T21:16:04.042 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:19.0.0-3212.g8254cb5a.el9.x86_64 23/34 2024-04-23T21:16:04.078 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-23T21:16:04.564 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-04-23T21:16:04.599 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:19.0.0-3212.g8254cb5a.el9.x86_64 26/34 2024-04-23T21:16:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:19.0.0-3212.g8254cb5a.el9.x86_64 26/34 2024-04-23T21:16:04.739 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:19.0.0-3212.g8254cb5a.el9.x86_64 27/34 2024-04-23T21:16:04.768 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:19.0.0-3212.g8254cb5a.el9.x8 28/34 2024-04-23T21:16:07.080 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-3212.g8254cb5a.el9.x86_64 29/34 2024-04-23T21:16:07.116 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:19.0.0-3212.g8254cb5a.el9.x86_64 29/34 2024-04-23T21:16:07.658 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-3212.g8254cb5a.el9.x86_64 29/34 2024-04-23T21:16:07.695 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:19.0.0-3212.g8254cb5a.el9.x86_64 30/34 2024-04-23T21:16:07.783 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:19.0.0-3212.g8254cb5a.el9.x86_64 30/34 2024-04-23T21:16:07.784 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T21:16:07.784 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T21:16:07.784 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:07.812 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:19.0.0-3212.g8254cb5a.el9.x86_64 31/34 2024-04-23T21:16:08.141 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3212.g8254cb5a.el9.x86_64 31/34 2024-04-23T21:16:08.141 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-04-23T21:16:08.141 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-04-23T21:16:08.141 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-04-23T21:16:08.141 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-04-23T21:16:08.141 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-04-23T21:16:08.142 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /root 2024-04-23T21:16:08.142 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-04-23T21:16:08.142 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:09.110 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:19.0.0-3212.g8254cb5a.el9.x86_64 32/34 2024-04-23T21:16:09.134 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3212.g8254cb5a.el9.x86_64 32/34 2024-04-23T21:16:09.135 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T21:16:09.135 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T21:16:09.135 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T21:16:09.135 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T21:16:09.135 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:09.136 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-23T21:16:09.222 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-23T21:16:09.222 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T21:16:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T21:16:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/34 2024-04-23T21:16:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:19.0.0-3212.g8254cb5a.el9.x86_64 2/34 2024-04-23T21:16:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:19.0.0-3212.g8254cb5a.el9.x86_64 3/34 2024-04-23T21:16:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:19.0.0-3212.g8254cb5a.el9.x86_64 4/34 2024-04-23T21:16:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:19.0.0-3212.g8254cb5a.el9.x86_64 5/34 2024-04-23T21:16:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:19.0.0-3212.g8254cb5a.el9.x86_6 6/34 2024-04-23T21:16:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:19.0.0-3212.g8254cb5a.el9.x86_64 7/34 2024-04-23T21:16:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3212.g8254cb5a.el9. 8/34 2024-04-23T21:16:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:19.0.0-3212.g8254cb5a.el9.x8 9/34 2024-04-23T21:16:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:19.0.0-3212.g8254cb5a.el9.x86_64 10/34 2024-04-23T21:16:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:19.0.0-3212.g8254cb5a.el9.x86_64 11/34 2024-04-23T21:16:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:19.0.0-3212.g8254cb5a.el9.x86_64 12/34 2024-04-23T21:16:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:19.0.0-3212.g8254cb5a.el9.x86_64 13/34 2024-04-23T21:16:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-23T21:16:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-23T21:16:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-23T21:16:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-23T21:16:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-23T21:16:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-23T21:16:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-23T21:16:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-23T21:16:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-23T21:16:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-23T21:16:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-04-23T21:16:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-04-23T21:16:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-23T21:16:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-23T21:16:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-04-23T21:16:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-23T21:16:10.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-23T21:16:10.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:19.0.0-3212.g8254cb5a.el9.x86_64 31/34 2024-04-23T21:16:10.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-23T21:16:10.356 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:19.0.0-3212.g8254cb5a.el9.x86_64 33/34 2024-04-23T21:16:11.144 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T21:16:11.144 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:11.144 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-04-23T21:16:11.145 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:11.145 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:11.145 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T21:16:11.145 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T21:16:11.145 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:11.145 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:11.145 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:11.145 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:11.145 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T21:16:11.145 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T21:16:11.146 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T21:16:11.146 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:11.146 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T21:16:11.146 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T21:16:11.146 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T21:16:11.146 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:11.146 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T21:16:11.146 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:11.146 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T21:16:11.146 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T21:16:11.147 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T21:16:11.147 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:11.147 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:11.147 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:11.147 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T21:16:11.147 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:11.147 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:11.147 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:11.147 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T21:16:11.147 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T21:16:11.147 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T21:16:11.148 INFO:teuthology.orchestra.run.smithi148.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T21:16:11.148 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T21:16:11.148 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:11.148 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:16:11.403 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-test 2024-04-23T21:16:11.937 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:28 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:16:12.058 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:16:12.058 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:16:12.058 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-04-23T21:16:12.059 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:16:12.059 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T21:16:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 49 M 2024-04-23T21:16:12.059 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-04-23T21:16:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 163 k 2024-04-23T21:16:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T21:16:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T21:16:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:12.059 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:16:12.059 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:16:12.059 INFO:teuthology.orchestra.run.smithi148.stdout:Install 4 Packages 2024-04-23T21:16:12.059 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:12.060 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 50 M 2024-04-23T21:16:12.060 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 211 M 2024-04-23T21:16:12.060 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:16:12.303 INFO:teuthology.orchestra.run.smithi148.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.7 MB/s | 305 kB 00:00 2024-04-23T21:16:12.345 INFO:teuthology.orchestra.run.smithi148.stdout:(2/4): libcephsqlite-19.0.0-3212.g8254cb5a.el9. 735 kB/s | 163 kB 00:00 2024-04-23T21:16:12.370 INFO:teuthology.orchestra.run.smithi148.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 949 kB/s | 64 kB 00:00 2024-04-23T21:16:14.732 INFO:teuthology.orchestra.run.smithi148.stdout:(4/4): ceph-test-19.0.0-3212.g8254cb5a.el9.x86_ 19 MB/s | 49 MB 00:02 2024-04-23T21:16:14.734 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:16:14.735 INFO:teuthology.orchestra.run.smithi148.stdout:Total 19 MB/s | 50 MB 00:02 2024-04-23T21:16:14.760 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:16:14.781 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:16:14.781 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:16:15.154 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:16:15.154 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:16:15.684 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:16:15.774 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-23T21:16:15.814 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-23T21:16:15.842 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:19.0.0-3212.g8254cb5a.el9.x86_64 3/4 2024-04-23T21:16:17.227 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3212.g8254cb5a.el9.x86_64 31/34 2024-04-23T21:16:17.228 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-04-23T21:16:17.228 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-04-23T21:16:17.228 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-04-23T21:16:17.228 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-04-23T21:16:17.228 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-04-23T21:16:17.228 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-04-23T21:16:17.228 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-04-23T21:16:17.228 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:18.205 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:19.0.0-3212.g8254cb5a.el9.x86_64 32/34 2024-04-23T21:16:18.229 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3212.g8254cb5a.el9.x86_64 32/34 2024-04-23T21:16:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T21:16:18.229 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T21:16:18.230 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T21:16:18.230 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T21:16:18.230 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:18.231 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-23T21:16:18.324 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-23T21:16:18.324 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T21:16:19.593 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T21:16:19.593 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/34 2024-04-23T21:16:19.593 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:19.0.0-3212.g8254cb5a.el9.x86_64 2/34 2024-04-23T21:16:19.593 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:19.0.0-3212.g8254cb5a.el9.x86_64 3/34 2024-04-23T21:16:19.593 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:19.0.0-3212.g8254cb5a.el9.x86_64 4/34 2024-04-23T21:16:19.594 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:19.0.0-3212.g8254cb5a.el9.x86_64 5/34 2024-04-23T21:16:19.594 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:19.0.0-3212.g8254cb5a.el9.x86_6 6/34 2024-04-23T21:16:19.594 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:19.0.0-3212.g8254cb5a.el9.x86_64 7/34 2024-04-23T21:16:19.594 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3212.g8254cb5a.el9. 8/34 2024-04-23T21:16:19.594 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:19.0.0-3212.g8254cb5a.el9.x8 9/34 2024-04-23T21:16:19.594 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:19.0.0-3212.g8254cb5a.el9.x86_64 10/34 2024-04-23T21:16:19.594 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:19.0.0-3212.g8254cb5a.el9.x86_64 11/34 2024-04-23T21:16:19.594 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:19.0.0-3212.g8254cb5a.el9.x86_64 12/34 2024-04-23T21:16:19.595 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:19.0.0-3212.g8254cb5a.el9.x86_64 13/34 2024-04-23T21:16:19.595 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-23T21:16:19.595 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-23T21:16:19.595 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-23T21:16:19.595 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-23T21:16:19.595 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-23T21:16:19.595 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-23T21:16:19.595 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-23T21:16:19.595 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-23T21:16:19.595 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-23T21:16:19.595 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-23T21:16:19.595 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-04-23T21:16:19.595 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-04-23T21:16:19.595 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-23T21:16:19.596 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-23T21:16:19.596 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-04-23T21:16:19.596 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-23T21:16:19.596 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-23T21:16:19.597 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:19.0.0-3212.g8254cb5a.el9.x86_64 31/34 2024-04-23T21:16:19.597 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-23T21:16:19.597 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:19.0.0-3212.g8254cb5a.el9.x86_64 33/34 2024-04-23T21:16:20.389 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-23T21:16:20.390 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:20.390 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-04-23T21:16:20.390 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:20.390 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:20.390 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-23T21:16:20.390 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T21:16:20.390 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:20.390 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:20.390 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:20.390 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:20.391 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T21:16:20.391 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T21:16:20.391 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T21:16:20.391 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:20.391 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T21:16:20.391 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T21:16:20.391 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T21:16:20.391 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:20.391 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T21:16:20.391 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:20.391 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T21:16:20.391 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T21:16:20.392 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T21:16:20.392 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:20.392 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:20.392 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:20.392 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T21:16:20.392 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:20.392 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:20.392 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:20.392 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T21:16:20.392 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T21:16:20.392 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T21:16:20.392 INFO:teuthology.orchestra.run.smithi150.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T21:16:20.392 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T21:16:20.392 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:20.393 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:16:20.466 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3212.g8254cb5a.el9.x86_64 3/4 2024-04-23T21:16:20.540 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:19.0.0-3212.g8254cb5a.el9.x86_64 4/4 2024-04-23T21:16:20.671 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-test 2024-04-23T21:16:21.209 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:31 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:16:21.333 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:16:21.333 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:16:21.333 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-23T21:16:21.333 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:16:21.333 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-23T21:16:21.334 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 49 M 2024-04-23T21:16:21.334 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-04-23T21:16:21.334 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 163 k 2024-04-23T21:16:21.334 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T21:16:21.334 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T21:16:21.334 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:21.334 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:16:21.334 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:16:21.334 INFO:teuthology.orchestra.run.smithi150.stdout:Install 4 Packages 2024-04-23T21:16:21.334 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:21.335 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 50 M 2024-04-23T21:16:21.335 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 211 M 2024-04-23T21:16:21.335 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:16:21.833 INFO:teuthology.orchestra.run.smithi150.stdout:(1/4): libcephsqlite-19.0.0-3212.g8254cb5a.el9. 462 kB/s | 163 kB 00:00 2024-04-23T21:16:22.582 INFO:teuthology.orchestra.run.smithi150.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 85 kB/s | 64 kB 00:00 2024-04-23T21:16:22.624 INFO:teuthology.orchestra.run.smithi150.stdout:(3/4): socat-1.7.4.1-5.el9.x86_64.rpm 266 kB/s | 305 kB 00:01 2024-04-23T21:16:22.881 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-test-2:19.0.0-3212.g8254cb5a.el9.x86_64 4/4 2024-04-23T21:16:22.881 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/4 2024-04-23T21:16:22.881 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:19.0.0-3212.g8254cb5a.el9.x86_64 2/4 2024-04-23T21:16:22.882 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-23T21:16:23.138 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-23T21:16:23.138 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:23.138 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T21:16:23.139 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:23.139 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:23.139 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T21:16:23.139 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T21:16:23.139 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:23.139 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:16:23.405 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph 2024-04-23T21:16:23.916 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:40 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:16:23.977 INFO:teuthology.orchestra.run.smithi150.stdout:(4/4): ceph-test-19.0.0-3212.g8254cb5a.el9.x86_ 20 MB/s | 49 MB 00:02 2024-04-23T21:16:23.978 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:16:23.979 INFO:teuthology.orchestra.run.smithi150.stdout:Total 19 MB/s | 50 MB 00:02 2024-04-23T21:16:24.005 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:16:24.026 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:16:24.026 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:16:24.044 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:16:24.047 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:16:24.047 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-04-23T21:16:24.047 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:16:24.047 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T21:16:24.047 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 6.6 k 2024-04-23T21:16:24.047 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-04-23T21:16:24.048 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 2.4 M 2024-04-23T21:16:24.048 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 1.5 M 2024-04-23T21:16:24.048 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 249 k 2024-04-23T21:16:24.048 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 4.7 M 2024-04-23T21:16:24.048 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 17 M 2024-04-23T21:16:24.048 INFO:teuthology.orchestra.run.smithi148.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T21:16:24.048 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T21:16:24.048 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T21:16:24.048 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T21:16:24.048 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T21:16:24.048 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T21:16:24.048 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T21:16:24.048 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T21:16:24.048 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T21:16:24.049 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T21:16:24.049 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections 2024-04-23T21:16:24.049 INFO:teuthology.orchestra.run.smithi148.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-23T21:16:24.049 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T21:16:24.049 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T21:16:24.049 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T21:16:24.049 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T21:16:24.049 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T21:16:24.049 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T21:16:24.049 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T21:16:24.049 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T21:16:24.049 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T21:16:24.049 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T21:16:24.049 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T21:16:24.050 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T21:16:24.050 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T21:16:24.050 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T21:16:24.050 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T21:16:24.050 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T21:16:24.050 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T21:16:24.050 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T21:16:24.050 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T21:16:24.050 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:24.050 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:16:24.051 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:16:24.051 INFO:teuthology.orchestra.run.smithi148.stdout:Install 36 Packages 2024-04-23T21:16:24.051 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:24.051 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 30 M 2024-04-23T21:16:24.051 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 107 M 2024-04-23T21:16:24.051 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:16:24.398 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:16:24.398 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:16:24.918 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:16:25.002 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-23T21:16:25.042 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-23T21:16:25.078 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:19.0.0-3212.g8254cb5a.el9.x86_64 3/4 2024-04-23T21:16:25.291 INFO:teuthology.orchestra.run.smithi148.stdout:(1/36): ceph-19.0.0-3212.g8254cb5a.el9.x86_64.r 38 kB/s | 6.6 kB 00:00 2024-04-23T21:16:25.500 INFO:teuthology.orchestra.run.smithi148.stdout:(2/36): ceph-mds-19.0.0-3212.g8254cb5a.el9.x86_ 6.2 MB/s | 2.4 MB 00:00 2024-04-23T21:16:25.950 INFO:teuthology.orchestra.run.smithi148.stdout:(3/36): ceph-mgr-19.0.0-3212.g8254cb5a.el9.x86_ 1.8 MB/s | 1.5 MB 00:00 2024-04-23T21:16:26.125 INFO:teuthology.orchestra.run.smithi148.stdout:(4/36): ceph-mgr-modules-core-19.0.0-3212.g8254 1.4 MB/s | 249 kB 00:00 2024-04-23T21:16:26.209 INFO:teuthology.orchestra.run.smithi148.stdout:(5/36): ceph-mon-19.0.0-3212.g8254cb5a.el9.x86_ 5.1 MB/s | 4.7 MB 00:00 2024-04-23T21:16:26.434 INFO:teuthology.orchestra.run.smithi148.stdout:(6/36): ceph-osd-19.0.0-3212.g8254cb5a.el9.x86_ 18 MB/s | 17 MB 00:00 2024-04-23T21:16:26.484 INFO:teuthology.orchestra.run.smithi148.stdout:(7/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 706 kB/s | 253 kB 00:00 2024-04-23T21:16:26.534 INFO:teuthology.orchestra.run.smithi148.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 2.7 MB/s | 135 kB 00:00 2024-04-23T21:16:26.593 INFO:teuthology.orchestra.run.smithi148.stdout:(9/36): python3-cryptography-36.0.1-4.el9.x86_6 3.2 MB/s | 1.2 MB 00:00 2024-04-23T21:16:26.610 INFO:teuthology.orchestra.run.smithi148.stdout:(10/36): python3-ply-3.11-14.el9.noarch.rpm 607 kB/s | 106 kB 00:00 2024-04-23T21:16:26.635 INFO:teuthology.orchestra.run.smithi148.stdout:(11/36): python3-requests-2.25.1-8.el9.noarch.r 1.2 MB/s | 125 kB 00:00 2024-04-23T21:16:26.660 INFO:teuthology.orchestra.run.smithi148.stdout:(12/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-04-23T21:16:26.786 INFO:teuthology.orchestra.run.smithi148.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 278 kB/s | 35 kB 00:00 2024-04-23T21:16:26.853 INFO:teuthology.orchestra.run.smithi148.stdout:(14/36): python3-mako-1.1.4-6.el9.noarch.rpm 792 kB/s | 172 kB 00:00 2024-04-23T21:16:26.878 INFO:teuthology.orchestra.run.smithi148.stdout:(15/36): lua-5.4.4-4.el9.x86_64.rpm 704 kB/s | 188 kB 00:00 2024-04-23T21:16:26.895 INFO:teuthology.orchestra.run.smithi148.stdout:(16/36): python3-pytz-2021.1-5.el9.noarch.rpm 468 kB/s | 51 kB 00:00 2024-04-23T21:16:26.912 INFO:teuthology.orchestra.run.smithi148.stdout:(17/36): python3-toml-0.10.2-6.el9.noarch.rpm 708 kB/s | 42 kB 00:00 2024-04-23T21:16:27.004 INFO:teuthology.orchestra.run.smithi148.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-04-23T21:16:27.096 INFO:teuthology.orchestra.run.smithi148.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 237 kB/s | 43 kB 00:00 2024-04-23T21:16:27.121 INFO:teuthology.orchestra.run.smithi148.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 670 kB/s | 151 kB 00:00 2024-04-23T21:16:27.163 INFO:teuthology.orchestra.run.smithi148.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 257 kB/s | 11 kB 00:00 2024-04-23T21:16:27.205 INFO:teuthology.orchestra.run.smithi148.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 425 kB/s | 18 kB 00:00 2024-04-23T21:16:27.230 INFO:teuthology.orchestra.run.smithi148.stdout:(23/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 763 kB/s | 172 kB 00:00 2024-04-23T21:16:27.255 INFO:teuthology.orchestra.run.smithi148.stdout:(24/36): python3-cherrypy-18.6.1-2.el9.noarch.r 2.2 MB/s | 358 kB 00:00 2024-04-23T21:16:27.281 INFO:teuthology.orchestra.run.smithi148.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 306 kB/s | 23 kB 00:00 2024-04-23T21:16:27.306 INFO:teuthology.orchestra.run.smithi148.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-04-23T21:16:27.331 INFO:teuthology.orchestra.run.smithi148.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 260 kB/s | 20 kB 00:00 2024-04-23T21:16:27.357 INFO:teuthology.orchestra.run.smithi148.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 614 kB/s | 46 kB 00:00 2024-04-23T21:16:27.382 INFO:teuthology.orchestra.run.smithi148.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-04-23T21:16:27.407 INFO:teuthology.orchestra.run.smithi148.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-04-23T21:16:27.432 INFO:teuthology.orchestra.run.smithi148.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-04-23T21:16:27.457 INFO:teuthology.orchestra.run.smithi148.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-04-23T21:16:27.474 INFO:teuthology.orchestra.run.smithi148.stdout:(33/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-04-23T21:16:27.508 INFO:teuthology.orchestra.run.smithi148.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-23T21:16:27.533 INFO:teuthology.orchestra.run.smithi148.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 341 kB/s | 20 kB 00:00 2024-04-23T21:16:27.592 INFO:teuthology.orchestra.run.smithi148.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.1 MB/s | 427 kB 00:00 2024-04-23T21:16:27.596 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:16:27.596 INFO:teuthology.orchestra.run.smithi148.stdout:Total 8.5 MB/s | 30 MB 00:03 2024-04-23T21:16:27.730 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:16:27.770 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:16:27.770 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:16:28.067 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:16:28.067 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:16:29.308 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:16:29.378 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-23T21:16:29.402 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-23T21:16:29.430 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-23T21:16:29.502 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-23T21:16:29.546 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-23T21:16:29.579 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-23T21:16:29.672 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-23T21:16:29.681 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3212.g8254cb5a.el9.x86_64 3/4 2024-04-23T21:16:29.750 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-23T21:16:29.750 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:19.0.0-3212.g8254cb5a.el9.x86_64 4/4 2024-04-23T21:16:29.790 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-23T21:16:29.825 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-23T21:16:29.854 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-23T21:16:29.889 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-23T21:16:29.922 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-23T21:16:29.963 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-23T21:16:30.015 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-23T21:16:30.049 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-23T21:16:30.087 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-23T21:16:30.170 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-23T21:16:30.308 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-23T21:16:30.388 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-23T21:16:30.440 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-23T21:16:30.478 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-23T21:16:30.526 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-23T21:16:30.582 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-23T21:16:30.770 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-23T21:16:30.816 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-23T21:16:30.894 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-23T21:16:30.998 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-23T21:16:31.040 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-23T21:16:31.157 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-23T21:16:31.321 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3212.g8254cb5a.el9. 31/36 2024-04-23T21:16:31.350 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:19.0.0-3212.g8254cb5a.el9.x86_64 32/36 2024-04-23T21:16:31.375 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3212.g8254cb5a.el9.x86_64 32/36 2024-04-23T21:16:31.375 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T21:16:31.375 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T21:16:31.375 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T21:16:31.375 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T21:16:31.376 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-test-2:19.0.0-3212.g8254cb5a.el9.x86_64 4/4 2024-04-23T21:16:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/4 2024-04-23T21:16:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:19.0.0-3212.g8254cb5a.el9.x86_64 2/4 2024-04-23T21:16:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-23T21:16:32.465 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-23T21:16:32.465 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:32.465 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-23T21:16:32.465 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:32.465 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:32.465 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T21:16:32.465 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T21:16:32.465 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:32.465 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:16:32.752 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph 2024-04-23T21:16:32.826 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:19.0.0-3212.g8254cb5a.el9.x86_64 33/36 2024-04-23T21:16:32.852 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:19.0.0-3212.g8254cb5a.el9.x86_64 33/36 2024-04-23T21:16:32.852 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T21:16:32.852 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T21:16:32.852 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T21:16:32.852 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T21:16:32.852 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:33.278 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:43 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:16:33.293 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:19.0.0-3212.g8254cb5a.el9.x86_64 34/36 2024-04-23T21:16:33.316 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:19.0.0-3212.g8254cb5a.el9.x86_64 34/36 2024-04-23T21:16:33.316 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T21:16:33.316 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T21:16:33.316 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T21:16:33.316 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T21:16:33.316 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:33.405 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:16:33.407 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:16:33.407 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-23T21:16:33.407 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:16:33.407 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-23T21:16:33.408 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 6.6 k 2024-04-23T21:16:33.408 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-04-23T21:16:33.408 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 2.4 M 2024-04-23T21:16:33.408 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 1.5 M 2024-04-23T21:16:33.408 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 249 k 2024-04-23T21:16:33.408 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 4.7 M 2024-04-23T21:16:33.408 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 17 M 2024-04-23T21:16:33.408 INFO:teuthology.orchestra.run.smithi150.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T21:16:33.408 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T21:16:33.408 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T21:16:33.408 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T21:16:33.408 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T21:16:33.408 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T21:16:33.408 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T21:16:33.408 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T21:16:33.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T21:16:33.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T21:16:33.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections 2024-04-23T21:16:33.409 INFO:teuthology.orchestra.run.smithi150.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-23T21:16:33.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T21:16:33.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T21:16:33.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T21:16:33.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T21:16:33.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T21:16:33.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T21:16:33.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T21:16:33.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T21:16:33.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T21:16:33.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T21:16:33.410 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T21:16:33.410 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T21:16:33.410 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T21:16:33.410 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T21:16:33.410 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T21:16:33.410 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T21:16:33.410 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T21:16:33.410 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T21:16:33.410 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T21:16:33.410 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:33.410 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:16:33.410 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:16:33.410 INFO:teuthology.orchestra.run.smithi150.stdout:Install 36 Packages 2024-04-23T21:16:33.410 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:33.411 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 30 M 2024-04-23T21:16:33.411 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 107 M 2024-04-23T21:16:33.411 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:16:33.535 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:19.0.0-3212.g8254cb5a.el9.x86_64 35/36 2024-04-23T21:16:33.558 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:19.0.0-3212.g8254cb5a.el9.x86_64 35/36 2024-04-23T21:16:33.558 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T21:16:33.558 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T21:16:33.558 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T21:16:33.558 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T21:16:33.558 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:33.619 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:19.0.0-3212.g8254cb5a.el9.x86_64 36/36 2024-04-23T21:16:34.578 INFO:teuthology.orchestra.run.smithi150.stdout:(1/36): ceph-mgr-19.0.0-3212.g8254cb5a.el9.x86_ 4.4 MB/s | 1.5 MB 00:00 2024-04-23T21:16:34.853 INFO:teuthology.orchestra.run.smithi150.stdout:(2/36): ceph-mon-19.0.0-3212.g8254cb5a.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-04-23T21:16:34.912 INFO:teuthology.orchestra.run.smithi150.stdout:(3/36): ceph-19.0.0-3212.g8254cb5a.el9.x86_64.r 9.7 kB/s | 6.6 kB 00:00 2024-04-23T21:16:35.045 INFO:teuthology.orchestra.run.smithi150.stdout:(4/36): ceph-mds-19.0.0-3212.g8254cb5a.el9.x86_ 2.9 MB/s | 2.4 MB 00:00 2024-04-23T21:16:35.206 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-2:19.0.0-3212.g8254cb5a.el9.x86_64 36/36 2024-04-23T21:16:35.206 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/36 2024-04-23T21:16:35.206 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:19.0.0-3212.g8254cb5a.el9.x86_64 2/36 2024-04-23T21:16:35.206 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:19.0.0-3212.g8254cb5a.el9.x86_64 3/36 2024-04-23T21:16:35.206 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:19.0.0-3212.g8254cb5a.el9.x86_64 4/36 2024-04-23T21:16:35.206 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:19.0.0-3212.g8254cb5a.el9.x86_64 5/36 2024-04-23T21:16:35.206 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3212.g8254cb5a.el9. 6/36 2024-04-23T21:16:35.206 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-23T21:16:35.206 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-23T21:16:35.207 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-23T21:16:35.207 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-23T21:16:35.207 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-23T21:16:35.207 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-23T21:16:35.207 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-23T21:16:35.207 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-23T21:16:35.207 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-23T21:16:35.207 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-23T21:16:35.207 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-23T21:16:35.207 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-23T21:16:35.207 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-23T21:16:35.207 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-23T21:16:35.207 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-23T21:16:35.207 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-23T21:16:35.208 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-23T21:16:35.208 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-23T21:16:35.208 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-23T21:16:35.208 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-23T21:16:35.208 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-23T21:16:35.208 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-23T21:16:35.209 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-23T21:16:35.209 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-23T21:16:35.210 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-23T21:16:35.210 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-23T21:16:35.210 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-23T21:16:35.210 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-23T21:16:35.210 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-23T21:16:35.254 INFO:teuthology.orchestra.run.smithi150.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-04-23T21:16:35.413 INFO:teuthology.orchestra.run.smithi150.stdout:(6/36): python3-cryptography-36.0.1-4.el9.x86_6 7.7 MB/s | 1.2 MB 00:00 2024-04-23T21:16:35.471 INFO:teuthology.orchestra.run.smithi150.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-23T21:16:35.530 INFO:teuthology.orchestra.run.smithi150.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 2.3 MB/s | 135 kB 00:00 2024-04-23T21:16:35.589 INFO:teuthology.orchestra.run.smithi150.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 2.1 MB/s | 125 kB 00:00 2024-04-23T21:16:35.764 INFO:teuthology.orchestra.run.smithi150.stdout:(10/36): ceph-osd-19.0.0-3212.g8254cb5a.el9.x86 18 MB/s | 17 MB 00:00 2024-04-23T21:16:35.789 INFO:teuthology.orchestra.run.smithi150.stdout:(11/36): ceph-mgr-modules-core-19.0.0-3212.g825 284 kB/s | 249 kB 00:00 2024-04-23T21:16:35.814 INFO:teuthology.orchestra.run.smithi150.stdout:(12/36): python3-urllib3-1.26.5-5.el9.noarch.rp 952 kB/s | 215 kB 00:00 2024-04-23T21:16:35.898 INFO:teuthology.orchestra.run.smithi150.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 418 kB/s | 35 kB 00:00 2024-04-23T21:16:35.923 INFO:teuthology.orchestra.run.smithi150.stdout:(14/36): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-04-23T21:16:35.948 INFO:teuthology.orchestra.run.smithi150.stdout:(15/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-04-23T21:16:35.965 INFO:teuthology.orchestra.run.smithi150.stdout:(16/36): python3-pytz-2021.1-5.el9.noarch.rpm 759 kB/s | 51 kB 00:00 2024-04-23T21:16:35.982 INFO:teuthology.orchestra.run.smithi150.stdout:(17/36): python3-toml-0.10.2-6.el9.noarch.rpm 708 kB/s | 42 kB 00:00 2024-04-23T21:16:36.049 INFO:teuthology.orchestra.run.smithi150.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 221 kB/s | 22 kB 00:00 2024-04-23T21:16:36.112 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-23T21:16:36.112 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:36.112 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T21:16:36.112 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:36.112 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:36.112 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:36.113 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:16:36.113 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:36.113 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:36.113 INFO:teuthology.orchestra.run.smithi148.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T21:16:36.113 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T21:16:36.113 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T21:16:36.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T21:16:36.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T21:16:36.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T21:16:36.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T21:16:36.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T21:16:36.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T21:16:36.113 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T21:16:36.114 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T21:16:36.114 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T21:16:36.114 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T21:16:36.114 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T21:16:36.114 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T21:16:36.114 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T21:16:36.114 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T21:16:36.114 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T21:16:36.114 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T21:16:36.114 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T21:16:36.114 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T21:16:36.114 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T21:16:36.114 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T21:16:36.115 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T21:16:36.115 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T21:16:36.115 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T21:16:36.115 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T21:16:36.115 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T21:16:36.115 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T21:16:36.115 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T21:16:36.115 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:36.115 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:16:36.324 INFO:teuthology.orchestra.run.smithi150.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 127 kB/s | 43 kB 00:00 2024-04-23T21:16:36.416 INFO:teuthology.orchestra.run.smithi150.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 335 kB/s | 151 kB 00:00 2024-04-23T21:16:36.448 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-base 2024-04-23T21:16:36.475 INFO:teuthology.orchestra.run.smithi150.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 404 kB/s | 172 kB 00:00 2024-04-23T21:16:36.609 INFO:teuthology.orchestra.run.smithi150.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 56 kB/s | 11 kB 00:00 2024-04-23T21:16:36.750 INFO:teuthology.orchestra.run.smithi150.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 64 kB/s | 18 kB 00:00 2024-04-23T21:16:36.876 INFO:teuthology.orchestra.run.smithi150.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 87 kB/s | 23 kB 00:00 2024-04-23T21:16:36.926 INFO:teuthology.orchestra.run.smithi150.stdout:(25/36): python3-cherrypy-18.6.1-2.el9.noarch.r 596 kB/s | 358 kB 00:00 2024-04-23T21:16:36.969 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:53 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:16:37.018 INFO:teuthology.orchestra.run.smithi150.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 73 kB/s | 19 kB 00:00 2024-04-23T21:16:37.049 INFO:teuthology.orchestra.run.smithi148.stdout:Package ceph-base-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:16:37.090 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:16:37.091 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-04-23T21:16:37.091 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:16:37.135 INFO:teuthology.orchestra.run.smithi150.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 76 kB/s | 20 kB 00:00 2024-04-23T21:16:37.152 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install cephadm 2024-04-23T21:16:37.285 INFO:teuthology.orchestra.run.smithi150.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 129 kB/s | 46 kB 00:00 2024-04-23T21:16:37.360 INFO:teuthology.orchestra.run.smithi150.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 230 kB/s | 79 kB 00:00 2024-04-23T21:16:37.552 INFO:teuthology.orchestra.run.smithi150.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 61 kB/s | 16 kB 00:00 2024-04-23T21:16:37.666 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:54 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:16:37.669 INFO:teuthology.orchestra.run.smithi150.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 510 kB/s | 272 kB 00:00 2024-04-23T21:16:37.703 INFO:teuthology.orchestra.run.smithi150.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 262 kB/s | 90 kB 00:00 2024-04-23T21:16:37.788 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:16:37.788 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:16:37.788 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-04-23T21:16:37.788 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:16:37.788 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T21:16:37.788 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 762 k 2024-04-23T21:16:37.788 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:37.788 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:16:37.788 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:16:37.789 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-04-23T21:16:37.789 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:37.789 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 762 k 2024-04-23T21:16:37.789 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 769 k 2024-04-23T21:16:37.789 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:16:37.862 INFO:teuthology.orchestra.run.smithi150.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 116 kB/s | 36 kB 00:00 2024-04-23T21:16:38.128 INFO:teuthology.orchestra.run.smithi150.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 75 kB/s | 20 kB 00:00 2024-04-23T21:16:38.170 INFO:teuthology.orchestra.run.smithi150.stdout:(35/36): python3-webob-1.8.7-6.el9.noarch.rpm 458 kB/s | 230 kB 00:00 2024-04-23T21:16:38.287 INFO:teuthology.orchestra.run.smithi150.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 732 kB/s | 427 kB 00:00 2024-04-23T21:16:38.291 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:16:38.291 INFO:teuthology.orchestra.run.smithi150.stdout:Total 6.2 MB/s | 30 MB 00:04 2024-04-23T21:16:38.428 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:16:38.467 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:16:38.468 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:16:38.701 INFO:teuthology.orchestra.run.smithi148.stdout:cephadm-19.0.0-3212.g8254cb5a.el9.noarch.rpm 836 kB/s | 762 kB 00:00 2024-04-23T21:16:38.702 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:16:38.702 INFO:teuthology.orchestra.run.smithi148.stdout:Total 834 kB/s | 762 kB 00:00 2024-04-23T21:16:38.702 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:16:38.708 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:16:38.708 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:16:38.714 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:16:38.714 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:16:38.762 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:16:38.763 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:16:38.789 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:16:39.211 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 1/1 2024-04-23T21:16:39.297 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 1/1 2024-04-23T21:16:39.653 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 1/1 2024-04-23T21:16:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 1/1 2024-04-23T21:16:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:39.876 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T21:16:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:16:39.876 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:39.876 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:16:40.036 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:16:40.080 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-immutable-object-cache 2024-04-23T21:16:40.098 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-23T21:16:40.139 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-23T21:16:40.168 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-23T21:16:40.237 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-23T21:16:40.299 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-23T21:16:40.332 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-23T21:16:40.425 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-23T21:16:40.519 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-23T21:16:40.562 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-23T21:16:40.594 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-23T21:16:40.604 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:57 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:16:40.691 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-23T21:16:40.726 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-23T21:16:40.727 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:16:40.728 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:16:40.728 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-04-23T21:16:40.728 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:16:40.728 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T21:16:40.728 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 145 k 2024-04-23T21:16:40.728 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:40.728 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:16:40.728 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:16:40.729 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-04-23T21:16:40.729 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:40.729 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 145 k 2024-04-23T21:16:40.729 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 435 k 2024-04-23T21:16:40.729 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:16:40.759 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-23T21:16:40.800 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-23T21:16:40.843 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-23T21:16:40.878 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-23T21:16:40.915 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-23T21:16:40.989 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-23T21:16:41.126 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-23T21:16:41.231 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-23T21:16:41.285 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-23T21:16:41.323 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-23T21:16:41.373 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-23T21:16:41.428 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-23T21:16:41.474 INFO:teuthology.orchestra.run.smithi148.stdout:ceph-immutable-object-cache-19.0.0-3212.g8254cb 195 kB/s | 145 kB 00:00 2024-04-23T21:16:41.475 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:16:41.475 INFO:teuthology.orchestra.run.smithi148.stdout:Total 194 kB/s | 145 kB 00:00 2024-04-23T21:16:41.475 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:16:41.483 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:16:41.483 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:16:41.541 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:16:41.542 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:16:41.621 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-23T21:16:41.669 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-23T21:16:41.690 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:16:41.747 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-23T21:16:41.775 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3212.g8254cb5a. 1/1 2024-04-23T21:16:41.799 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3212.g8254cb5a. 1/1 2024-04-23T21:16:41.799 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T21:16:41.799 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T21:16:41.799 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:41.836 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-23T21:16:41.901 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-23T21:16:42.016 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-23T21:16:42.175 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3212.g8254cb5a.el9. 31/36 2024-04-23T21:16:42.221 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:19.0.0-3212.g8254cb5a.el9.x86_64 32/36 2024-04-23T21:16:42.246 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3212.g8254cb5a.el9.x86_64 32/36 2024-04-23T21:16:42.246 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T21:16:42.246 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T21:16:42.246 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T21:16:42.246 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T21:16:42.246 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:42.678 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3212.g8254cb5a. 1/1 2024-04-23T21:16:42.679 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:42.679 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T21:16:42.679 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:42.679 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:42.679 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:16:42.901 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr 2024-04-23T21:16:43.415 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:00 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:16:43.495 INFO:teuthology.orchestra.run.smithi148.stdout:Package ceph-mgr-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:16:43.540 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:16:43.541 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-04-23T21:16:43.541 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:16:43.600 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-dashboard 2024-04-23T21:16:43.688 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:19.0.0-3212.g8254cb5a.el9.x86_64 33/36 2024-04-23T21:16:43.715 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:19.0.0-3212.g8254cb5a.el9.x86_64 33/36 2024-04-23T21:16:43.716 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T21:16:43.716 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T21:16:43.716 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T21:16:43.716 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T21:16:43.716 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:44.099 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:01 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:16:44.156 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:19.0.0-3212.g8254cb5a.el9.x86_64 34/36 2024-04-23T21:16:44.178 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:19.0.0-3212.g8254cb5a.el9.x86_64 34/36 2024-04-23T21:16:44.178 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T21:16:44.178 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T21:16:44.178 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T21:16:44.179 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T21:16:44.179 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:44.219 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:16:44.221 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:16:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-04-23T21:16:44.221 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:16:44.221 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T21:16:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 3.5 M 2024-04-23T21:16:44.221 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-04-23T21:16:44.221 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T21:16:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 34 k 2024-04-23T21:16:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 16 k 2024-04-23T21:16:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T21:16:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T21:16:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T21:16:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T21:16:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T21:16:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T21:16:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T21:16:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T21:16:44.222 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:44.222 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:16:44.223 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:16:44.223 INFO:teuthology.orchestra.run.smithi148.stdout:Install 12 Packages 2024-04-23T21:16:44.223 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:44.223 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 8.6 M 2024-04-23T21:16:44.223 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 104 M 2024-04-23T21:16:44.223 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:16:44.398 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:19.0.0-3212.g8254cb5a.el9.x86_64 35/36 2024-04-23T21:16:44.421 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:19.0.0-3212.g8254cb5a.el9.x86_64 35/36 2024-04-23T21:16:44.421 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T21:16:44.421 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T21:16:44.421 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T21:16:44.421 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T21:16:44.421 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:44.475 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:19.0.0-3212.g8254cb5a.el9.x86_64 36/36 2024-04-23T21:16:45.052 INFO:teuthology.orchestra.run.smithi148.stdout:(1/12): ceph-grafana-dashboards-19.0.0-3212.g82 99 kB/s | 34 kB 00:00 2024-04-23T21:16:45.086 INFO:teuthology.orchestra.run.smithi148.stdout:(2/12): ceph-prometheus-alerts-19.0.0-3212.g825 44 kB/s | 16 kB 00:00 2024-04-23T21:16:45.211 INFO:teuthology.orchestra.run.smithi148.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.1 MB/s | 269 kB 00:00 2024-04-23T21:16:45.245 INFO:teuthology.orchestra.run.smithi148.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.2 MB/s | 1.0 MB 00:00 2024-04-23T21:16:45.462 INFO:teuthology.orchestra.run.smithi148.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.5 MB/s | 548 kB 00:00 2024-04-23T21:16:45.495 INFO:teuthology.orchestra.run.smithi148.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 588 kB/s | 19 kB 00:00 2024-04-23T21:16:45.529 INFO:teuthology.orchestra.run.smithi148.stdout:(7/12): protobuf-compiler-3.14.0-13.el9.x86_64. 2.7 MB/s | 863 kB 00:00 2024-04-23T21:16:45.612 INFO:teuthology.orchestra.run.smithi148.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 17 MB/s | 2.0 MB 00:00 2024-04-23T21:16:45.655 INFO:teuthology.orchestra.run.smithi148.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 754 kB/s | 31 kB 00:00 2024-04-23T21:16:45.696 INFO:teuthology.orchestra.run.smithi148.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 4.4 MB/s | 188 kB 00:00 2024-04-23T21:16:45.746 INFO:teuthology.orchestra.run.smithi148.stdout:(11/12): python3-grpcio-tools-1.46.7-10.el9.x86 665 kB/s | 144 kB 00:00 2024-04-23T21:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-2:19.0.0-3212.g8254cb5a.el9.x86_64 36/36 2024-04-23T21:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/36 2024-04-23T21:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:19.0.0-3212.g8254cb5a.el9.x86_64 2/36 2024-04-23T21:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:19.0.0-3212.g8254cb5a.el9.x86_64 3/36 2024-04-23T21:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:19.0.0-3212.g8254cb5a.el9.x86_64 4/36 2024-04-23T21:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:19.0.0-3212.g8254cb5a.el9.x86_64 5/36 2024-04-23T21:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3212.g8254cb5a.el9. 6/36 2024-04-23T21:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-23T21:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-23T21:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-23T21:16:46.352 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-23T21:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-23T21:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-23T21:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-23T21:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-23T21:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-23T21:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-23T21:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-23T21:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-23T21:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-23T21:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-23T21:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-23T21:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-23T21:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-23T21:16:46.353 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-23T21:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-23T21:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-23T21:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-23T21:16:46.354 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-23T21:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-23T21:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-23T21:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-23T21:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-23T21:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-23T21:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-23T21:16:46.355 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-23T21:16:46.356 INFO:teuthology.orchestra.run.smithi148.stdout:(12/12): ceph-mgr-dashboard-19.0.0-3212.g8254cb 2.1 MB/s | 3.5 MB 00:01 2024-04-23T21:16:46.358 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:16:46.358 INFO:teuthology.orchestra.run.smithi148.stdout:Total 4.0 MB/s | 8.6 MB 00:02 2024-04-23T21:16:46.405 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:16:46.422 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:16:46.422 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:16:46.576 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:16:46.577 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:16:47.101 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:16:47.181 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-04-23T21:16:47.224 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-04-23T21:16:47.240 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-23T21:16:47.240 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:47.240 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-23T21:16:47.240 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:47.240 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:47.240 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:47.240 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:16:47.241 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:47.241 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:47.241 INFO:teuthology.orchestra.run.smithi150.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T21:16:47.241 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T21:16:47.241 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T21:16:47.241 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T21:16:47.241 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T21:16:47.241 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T21:16:47.241 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T21:16:47.241 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T21:16:47.241 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T21:16:47.241 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T21:16:47.241 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T21:16:47.241 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T21:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T21:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T21:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T21:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T21:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T21:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T21:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T21:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T21:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T21:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T21:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T21:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T21:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T21:16:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T21:16:47.243 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T21:16:47.243 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T21:16:47.243 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T21:16:47.243 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T21:16:47.243 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:47.243 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:16:47.270 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-04-23T21:16:47.326 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-04-23T21:16:47.402 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-04-23T21:16:47.544 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-04-23T21:16:47.564 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-base 2024-04-23T21:16:47.637 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-04-23T21:16:47.681 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-04-23T21:16:47.709 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-04-23T21:16:47.741 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3212.g8254cb5a.el9 10/12 2024-04-23T21:16:48.080 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:58 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:16:48.158 INFO:teuthology.orchestra.run.smithi150.stdout:Package ceph-base-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:16:48.200 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:16:48.201 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-04-23T21:16:48.201 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:16:48.255 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install cephadm 2024-04-23T21:16:48.763 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:58 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:16:48.790 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3212.g8254cb5a.el 11/12 2024-04-23T21:16:48.825 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3212.g8254cb5a.el9.noa 12/12 2024-04-23T21:16:48.884 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:16:48.885 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:16:48.885 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-23T21:16:48.885 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:16:48.885 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-23T21:16:48.885 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 762 k 2024-04-23T21:16:48.885 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:48.885 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:16:48.885 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:16:48.885 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-04-23T21:16:48.886 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:48.886 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 762 k 2024-04-23T21:16:48.886 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 769 k 2024-04-23T21:16:48.886 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:16:49.288 INFO:teuthology.orchestra.run.smithi150.stdout:cephadm-19.0.0-3212.g8254cb5a.el9.noarch.rpm 1.8 MB/s | 762 kB 00:00 2024-04-23T21:16:49.289 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:16:49.289 INFO:teuthology.orchestra.run.smithi150.stdout:Total 1.8 MB/s | 762 kB 00:00 2024-04-23T21:16:49.289 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:16:49.294 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:16:49.294 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:16:49.300 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:16:49.300 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:16:49.369 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:16:49.738 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 1/1 2024-04-23T21:16:49.823 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 1/1 2024-04-23T21:16:50.151 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 1/1 2024-04-23T21:16:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3212.g8254cb5a.el9.noa 12/12 2024-04-23T21:16:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3212.g8254cb5a.el 1/12 2024-04-23T21:16:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3212.g8254cb5a.el9.noa 2/12 2024-04-23T21:16:50.152 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3212.g8254cb5a.el9 3/12 2024-04-23T21:16:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-04-23T21:16:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-04-23T21:16:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-04-23T21:16:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-04-23T21:16:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-04-23T21:16:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-04-23T21:16:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-04-23T21:16:50.153 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-04-23T21:16:50.399 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 1/1 2024-04-23T21:16:50.400 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:50.400 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-23T21:16:50.400 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:16:50.400 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:50.400 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:16:50.435 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-04-23T21:16:50.435 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:50.435 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T21:16:50.435 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T21:16:50.435 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:16:50.435 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:16:50.435 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:16:50.435 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T21:16:50.435 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T21:16:50.436 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T21:16:50.436 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T21:16:50.436 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T21:16:50.436 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T21:16:50.436 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T21:16:50.436 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T21:16:50.436 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:50.436 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:16:50.574 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-immutable-object-cache 2024-04-23T21:16:50.695 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-23T21:16:51.101 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:01 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:16:51.200 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:08 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:16:51.224 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:16:51.224 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:16:51.225 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-04-23T21:16:51.225 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:16:51.225 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-23T21:16:51.225 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 145 k 2024-04-23T21:16:51.225 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:51.225 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:16:51.225 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:16:51.225 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-04-23T21:16:51.225 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:51.226 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 145 k 2024-04-23T21:16:51.226 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 435 k 2024-04-23T21:16:51.226 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:16:51.321 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:16:51.322 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-04-23T21:16:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-04-23T21:16:51.323 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-04-23T21:16:51.323 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T21:16:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 7.4 M 2024-04-23T21:16:51.323 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-04-23T21:16:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T21:16:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T21:16:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T21:16:51.323 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T21:16:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T21:16:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T21:16:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T21:16:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T21:16:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T21:16:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T21:16:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T21:16:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:51.324 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:16:51.324 INFO:teuthology.orchestra.run.smithi148.stdout:======================================================================================= 2024-04-23T21:16:51.324 INFO:teuthology.orchestra.run.smithi148.stdout:Install 12 Packages 2024-04-23T21:16:51.324 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:16:51.325 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 38 M 2024-04-23T21:16:51.325 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 209 M 2024-04-23T21:16:51.325 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:16:51.838 INFO:teuthology.orchestra.run.smithi148.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-04-23T21:16:51.879 INFO:teuthology.orchestra.run.smithi148.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 803 kB 00:00 2024-04-23T21:16:51.980 INFO:teuthology.orchestra.run.smithi148.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 231 kB/s | 33 kB 00:00 2024-04-23T21:16:52.038 INFO:teuthology.orchestra.run.smithi148.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 312 kB/s | 18 kB 00:00 2024-04-23T21:16:52.097 INFO:teuthology.orchestra.run.smithi148.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 604 kB/s | 35 kB 00:00 2024-04-23T21:16:52.289 INFO:teuthology.orchestra.run.smithi148.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.3 MB/s | 3.0 MB 00:00 2024-04-23T21:16:52.356 INFO:teuthology.orchestra.run.smithi148.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-23T21:16:52.564 INFO:teuthology.orchestra.run.smithi148.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-04-23T21:16:52.631 INFO:teuthology.orchestra.run.smithi148.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 3.7 MB/s | 250 kB 00:00 2024-04-23T21:16:52.749 INFO:teuthology.orchestra.run.smithi148.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 7.3 MB/s | 4.7 MB 00:00 2024-04-23T21:16:53.135 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-immutable-object-cache-19.0.0-3212.g8254cb 76 kB/s | 145 kB 00:01 2024-04-23T21:16:53.136 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:16:53.136 INFO:teuthology.orchestra.run.smithi150.stdout:Total 76 kB/s | 145 kB 00:01 2024-04-23T21:16:53.136 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:16:53.146 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:16:53.146 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:16:53.148 INFO:teuthology.orchestra.run.smithi148.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-04-23T21:16:53.206 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:16:53.206 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:16:53.365 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:16:53.445 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3212.g8254cb5a. 1/1 2024-04-23T21:16:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3212.g8254cb5a. 1/1 2024-04-23T21:16:53.469 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T21:16:53.469 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T21:16:53.469 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:53.632 INFO:teuthology.orchestra.run.smithi148.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-3 3.8 MB/s | 7.4 MB 00:01 2024-04-23T21:16:53.633 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:16:53.633 INFO:teuthology.orchestra.run.smithi148.stdout:Total 16 MB/s | 38 MB 00:02 2024-04-23T21:16:53.997 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:16:54.023 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:16:54.023 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:16:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3212.g8254cb5a. 1/1 2024-04-23T21:16:54.381 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:54.381 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-23T21:16:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:16:54.382 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:54.382 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:16:54.448 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:16:54.448 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:16:54.553 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr 2024-04-23T21:16:55.078 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:05 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:16:55.159 INFO:teuthology.orchestra.run.smithi150.stdout:Package ceph-mgr-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:16:55.202 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:16:55.203 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-04-23T21:16:55.204 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:16:55.260 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-dashboard 2024-04-23T21:16:55.291 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:16:55.354 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-23T21:16:55.406 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-23T21:16:55.513 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-23T21:16:55.557 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-23T21:16:55.737 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-23T21:16:55.761 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:05 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:16:55.764 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-23T21:16:55.842 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-23T21:16:55.883 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:16:55.884 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:16:55.884 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-23T21:16:55.884 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:16:55.885 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-23T21:16:55.885 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 3.5 M 2024-04-23T21:16:55.885 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-04-23T21:16:55.885 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T21:16:55.885 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 34 k 2024-04-23T21:16:55.885 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 16 k 2024-04-23T21:16:55.885 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T21:16:55.885 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T21:16:55.885 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T21:16:55.885 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T21:16:55.885 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T21:16:55.885 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T21:16:55.885 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T21:16:55.885 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T21:16:55.886 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:55.886 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:16:55.886 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:16:55.886 INFO:teuthology.orchestra.run.smithi150.stdout:Install 12 Packages 2024-04-23T21:16:55.886 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:16:55.887 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 8.6 M 2024-04-23T21:16:55.887 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 104 M 2024-04-23T21:16:55.887 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:16:56.475 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-23T21:16:56.585 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-23T21:16:56.706 INFO:teuthology.orchestra.run.smithi150.stdout:(1/12): ceph-prometheus-alerts-19.0.0-3212.g825 89 kB/s | 16 kB 00:00 2024-04-23T21:16:56.756 INFO:teuthology.orchestra.run.smithi150.stdout:(2/12): ceph-grafana-dashboards-19.0.0-3212.g82 145 kB/s | 34 kB 00:00 2024-04-23T21:16:56.973 INFO:teuthology.orchestra.run.smithi150.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.2 MB/s | 269 kB 00:00 2024-04-23T21:16:57.048 INFO:teuthology.orchestra.run.smithi150.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.9 MB/s | 1.0 MB 00:00 2024-04-23T21:16:57.407 INFO:teuthology.orchestra.run.smithi150.stdout:(5/12): ceph-mgr-dashboard-19.0.0-3212.g8254cb5 4.0 MB/s | 3.5 MB 00:00 2024-04-23T21:16:57.432 INFO:teuthology.orchestra.run.smithi150.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.4 MB/s | 548 kB 00:00 2024-04-23T21:16:57.599 INFO:teuthology.orchestra.run.smithi150.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 101 kB/s | 19 kB 00:00 2024-04-23T21:16:57.640 INFO:teuthology.orchestra.run.smithi150.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 9.8 MB/s | 2.0 MB 00:00 2024-04-23T21:16:57.699 INFO:teuthology.orchestra.run.smithi150.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 527 kB/s | 31 kB 00:00 2024-04-23T21:16:57.716 INFO:teuthology.orchestra.run.smithi150.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 1.2 MB/s | 144 kB 00:00 2024-04-23T21:16:57.758 INFO:teuthology.orchestra.run.smithi150.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.1 MB/s | 188 kB 00:00 2024-04-23T21:16:57.801 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-23T21:16:58.667 INFO:teuthology.orchestra.run.smithi150.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 510 kB/s | 863 kB 00:01 2024-04-23T21:16:58.672 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:16:58.672 INFO:teuthology.orchestra.run.smithi150.stdout:Total 3.1 MB/s | 8.6 MB 00:02 2024-04-23T21:16:58.721 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:16:58.739 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:16:58.739 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:16:58.847 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-23T21:16:58.880 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3212.g8254c 12/12 2024-04-23T21:16:58.897 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:16:58.900 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:16:59.423 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:16:59.492 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-04-23T21:16:59.527 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-04-23T21:16:59.574 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-04-23T21:16:59.605 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-04-23T21:16:59.680 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-04-23T21:16:59.815 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-04-23T21:16:59.899 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-04-23T21:16:59.942 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-04-23T21:16:59.971 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-04-23T21:17:00.003 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3212.g8254cb5a.el9 10/12 2024-04-23T21:17:01.058 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3212.g8254cb5a.el 11/12 2024-04-23T21:17:01.103 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3212.g8254cb5a.el9.noa 12/12 2024-04-23T21:17:01.518 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3212.g8254c 12/12 2024-04-23T21:17:01.518 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3212.g8254c 1/12 2024-04-23T21:17:01.518 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-23T21:17:01.518 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-23T21:17:01.518 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-23T21:17:01.518 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-23T21:17:01.518 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-23T21:17:01.518 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-23T21:17:01.518 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-23T21:17:01.518 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-23T21:17:01.518 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-23T21:17:01.519 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-23T21:17:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-23T21:17:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:01.895 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T21:17:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:17:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T21:17:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T21:17:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T21:17:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T21:17:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T21:17:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T21:17:01.895 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T21:17:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T21:17:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T21:17:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T21:17:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T21:17:01.896 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:01.896 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:02.254 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-rook 2024-04-23T21:17:02.511 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3212.g8254cb5a.el9.noa 12/12 2024-04-23T21:17:02.511 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3212.g8254cb5a.el 1/12 2024-04-23T21:17:02.511 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3212.g8254cb5a.el9.noa 2/12 2024-04-23T21:17:02.511 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3212.g8254cb5a.el9 3/12 2024-04-23T21:17:02.511 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-04-23T21:17:02.512 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-04-23T21:17:02.512 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-04-23T21:17:02.512 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-04-23T21:17:02.512 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-04-23T21:17:02.512 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-04-23T21:17:02.512 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-04-23T21:17:02.512 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-04-23T21:17:02.774 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:19 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-04-23T21:17:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:02.796 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-23T21:17:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T21:17:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:17:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:17:02.796 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:17:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T21:17:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T21:17:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T21:17:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T21:17:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T21:17:02.797 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T21:17:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T21:17:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T21:17:02.798 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:02.798 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:02.895 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:02.896 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:02.896 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-04-23T21:17:02.896 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:02.896 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T21:17:02.896 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 49 k 2024-04-23T21:17:02.896 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-04-23T21:17:02.896 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T21:17:02.897 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T21:17:02.897 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T21:17:02.897 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T21:17:02.897 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T21:17:02.897 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T21:17:02.897 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T21:17:02.897 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T21:17:02.897 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T21:17:02.897 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T21:17:02.897 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T21:17:02.897 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T21:17:02.897 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:02.897 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:17:02.897 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:02.898 INFO:teuthology.orchestra.run.smithi148.stdout:Install 13 Packages 2024-04-23T21:17:02.898 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:02.898 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 2.2 M 2024-04-23T21:17:02.898 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 26 M 2024-04-23T21:17:02.898 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:17:03.061 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-23T21:17:03.571 INFO:teuthology.orchestra.run.smithi148.stdout:(1/13): ceph-mgr-rook-19.0.0-3212.g8254cb5a.el9 186 kB/s | 49 kB 00:00 2024-04-23T21:17:03.588 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:13 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:03.596 INFO:teuthology.orchestra.run.smithi148.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 88 kB/s | 26 kB 00:00 2024-04-23T21:17:03.621 INFO:teuthology.orchestra.run.smithi148.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 61 kB/s | 19 kB 00:00 2024-04-23T21:17:03.719 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:03.721 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-23T21:17:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-23T21:17:03.721 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-23T21:17:03.721 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-23T21:17:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 7.4 M 2024-04-23T21:17:03.721 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-04-23T21:17:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T21:17:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T21:17:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T21:17:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T21:17:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T21:17:03.721 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T21:17:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T21:17:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T21:17:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T21:17:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T21:17:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T21:17:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:03.722 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:17:03.722 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-23T21:17:03.722 INFO:teuthology.orchestra.run.smithi150.stdout:Install 12 Packages 2024-04-23T21:17:03.722 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:03.723 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 38 M 2024-04-23T21:17:03.723 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 209 M 2024-04-23T21:17:03.723 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:17:03.972 INFO:teuthology.orchestra.run.smithi148.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 454 kB/s | 159 kB 00:00 2024-04-23T21:17:03.997 INFO:teuthology.orchestra.run.smithi148.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 522 kB/s | 222 kB 00:00 2024-04-23T21:17:04.089 INFO:teuthology.orchestra.run.smithi148.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 458 kB/s | 54 kB 00:00 2024-04-23T21:17:04.206 INFO:teuthology.orchestra.run.smithi148.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 477 kB/s | 279 kB 00:00 2024-04-23T21:17:04.256 INFO:teuthology.orchestra.run.smithi148.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 124 kB/s | 32 kB 00:00 2024-04-23T21:17:04.315 INFO:teuthology.orchestra.run.smithi148.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 66 kB/s | 15 kB 00:00 2024-04-23T21:17:04.484 INFO:teuthology.orchestra.run.smithi150.stdout:(1/12): ceph-mgr-diskprediction-local-19.0.0-32 13 MB/s | 7.4 MB 00:00 2024-04-23T21:17:04.515 INFO:teuthology.orchestra.run.smithi148.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 295 kB/s | 59 kB 00:00 2024-04-23T21:17:04.534 INFO:teuthology.orchestra.run.smithi150.stdout:(2/12): libquadmath-11.4.1-3.el9.x86_64.rpm 310 kB/s | 195 kB 00:00 2024-04-23T21:17:04.649 INFO:teuthology.orchestra.run.smithi148.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 2.6 MB/s | 1.0 MB 00:00 2024-04-23T21:17:04.659 INFO:teuthology.orchestra.run.smithi150.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 187 kB/s | 33 kB 00:00 2024-04-23T21:17:04.674 INFO:teuthology.orchestra.run.smithi148.stdout:(12/13): python3-google-auth-2.29.0-1.el9.noarc 440 kB/s | 206 kB 00:00 2024-04-23T21:17:04.702 INFO:teuthology.orchestra.run.smithi148.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 488 kB/s | 90 kB 00:00 2024-04-23T21:17:04.702 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:04.703 INFO:teuthology.orchestra.run.smithi148.stdout:Total 1.2 MB/s | 2.2 MB 00:01 2024-04-23T21:17:04.717 INFO:teuthology.orchestra.run.smithi150.stdout:(4/12): libgfortran-11.4.1-3.el9.x86_64.rpm 987 kB/s | 803 kB 00:00 2024-04-23T21:17:04.735 INFO:teuthology.orchestra.run.smithi150.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 241 kB/s | 18 kB 00:00 2024-04-23T21:17:04.768 INFO:teuthology.orchestra.run.smithi150.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 700 kB/s | 35 kB 00:00 2024-04-23T21:17:04.777 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:17:04.793 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:17:04.793 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:17:04.877 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:17:04.878 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:17:04.894 INFO:teuthology.orchestra.run.smithi150.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-04-23T21:17:05.019 INFO:teuthology.orchestra.run.smithi150.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 6.2 MB/s | 3.0 MB 00:00 2024-04-23T21:17:05.086 INFO:teuthology.orchestra.run.smithi150.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 3.7 MB/s | 250 kB 00:00 2024-04-23T21:17:05.295 INFO:teuthology.orchestra.run.smithi150.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 8.5 MB/s | 4.7 MB 00:00 2024-04-23T21:17:05.361 INFO:teuthology.orchestra.run.smithi150.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-04-23T21:17:05.457 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:17:05.630 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-23T21:17:05.683 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-23T21:17:05.739 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-23T21:17:05.785 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-23T21:17:05.826 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-23T21:17:05.931 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-23T21:17:05.931 INFO:teuthology.orchestra.run.smithi150.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-04-23T21:17:05.932 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:05.932 INFO:teuthology.orchestra.run.smithi150.stdout:Total 17 MB/s | 38 MB 00:02 2024-04-23T21:17:06.038 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-23T21:17:06.086 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-23T21:17:06.276 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:17:06.303 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:17:06.303 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:17:06.713 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:17:06.713 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:17:06.798 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-23T21:17:06.850 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-23T21:17:06.892 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-23T21:17:06.939 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-23T21:17:06.961 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:19.0.0-3212.g8254cb5a.el9.noarch 13/13 2024-04-23T21:17:07.522 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:17:07.577 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3212.g8254cb5a.el9.noarch 13/13 2024-04-23T21:17:07.577 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3212.g8254cb5a.el9.noarch 1/13 2024-04-23T21:17:07.577 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-23T21:17:07.577 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-23T21:17:07.577 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-23T21:17:07.577 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-23T21:17:07.577 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-23T21:17:07.577 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-23T21:17:07.577 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-23T21:17:07.577 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-23T21:17:07.578 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-23T21:17:07.578 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-23T21:17:07.578 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-23T21:17:07.583 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-23T21:17:07.627 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-23T21:17:07.725 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-23T21:17:07.769 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-23T21:17:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-23T21:17:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:07.876 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T21:17:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:17:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T21:17:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T21:17:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T21:17:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T21:17:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T21:17:07.876 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T21:17:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T21:17:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T21:17:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T21:17:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T21:17:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T21:17:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T21:17:07.877 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:07.877 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:07.948 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-23T21:17:07.977 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-23T21:17:08.045 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-23T21:17:08.119 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-mgr-cephadm 2024-04-23T21:17:08.632 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:25 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:08.680 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-23T21:17:08.752 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:08.753 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:08.753 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-04-23T21:17:08.753 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:08.753 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T21:17:08.753 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 142 k 2024-04-23T21:17:08.753 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-04-23T21:17:08.753 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T21:17:08.753 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T21:17:08.753 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T21:17:08.753 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T21:17:08.754 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T21:17:08.754 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:08.754 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:17:08.754 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:08.754 INFO:teuthology.orchestra.run.smithi148.stdout:Install 6 Packages 2024-04-23T21:17:08.754 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:08.754 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 7.0 M 2024-04-23T21:17:08.755 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 33 M 2024-04-23T21:17:08.755 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:17:08.799 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-23T21:17:09.459 INFO:teuthology.orchestra.run.smithi148.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-04-23T21:17:09.667 INFO:teuthology.orchestra.run.smithi148.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.6 MB/s | 548 kB 00:00 2024-04-23T21:17:09.717 INFO:teuthology.orchestra.run.smithi148.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-04-23T21:17:09.768 INFO:teuthology.orchestra.run.smithi148.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 1.0 MB/s | 51 kB 00:00 2024-04-23T21:17:09.810 INFO:teuthology.orchestra.run.smithi148.stdout:(5/6): ceph-mgr-cephadm-19.0.0-3212.g8254cb5a.e 247 kB/s | 142 kB 00:00 2024-04-23T21:17:10.030 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-23T21:17:10.431 INFO:teuthology.orchestra.run.smithi148.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.0 MB/s | 6.0 MB 00:01 2024-04-23T21:17:10.431 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:10.431 INFO:teuthology.orchestra.run.smithi148.stdout:Total 4.2 MB/s | 7.0 MB 00:01 2024-04-23T21:17:10.498 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:17:10.509 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:17:10.509 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:17:10.606 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:17:10.608 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:17:10.903 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:17:11.065 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-23T21:17:11.066 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-23T21:17:11.093 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3212.g8254c 12/12 2024-04-23T21:17:11.112 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-23T21:17:11.628 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-23T21:17:11.714 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-23T21:17:11.785 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-23T21:17:11.815 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 6/6 2024-04-23T21:17:12.517 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 6/6 2024-04-23T21:17:12.517 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 1/6 2024-04-23T21:17:12.517 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-23T21:17:12.517 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-23T21:17:12.517 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-23T21:17:12.518 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-23T21:17:12.791 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-23T21:17:12.791 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:12.791 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T21:17:12.791 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:17:12.791 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T21:17:12.791 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T21:17:12.791 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T21:17:12.791 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T21:17:12.791 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T21:17:12.792 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:12.792 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:13.006 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-fuse 2024-04-23T21:17:13.518 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:30 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:13.639 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:13.640 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-04-23T21:17:13.640 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:13.640 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T21:17:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 890 k 2024-04-23T21:17:13.640 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-04-23T21:17:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T21:17:13.640 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:13.640 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:17:13.640 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:13.641 INFO:teuthology.orchestra.run.smithi148.stdout:Install 2 Packages 2024-04-23T21:17:13.641 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:13.641 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 970 k 2024-04-23T21:17:13.641 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 2.8 M 2024-04-23T21:17:13.641 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:17:13.738 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3212.g8254c 12/12 2024-04-23T21:17:13.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3212.g8254c 1/12 2024-04-23T21:17:13.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-23T21:17:13.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-23T21:17:13.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-23T21:17:13.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-23T21:17:13.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-23T21:17:13.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-23T21:17:13.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-23T21:17:13.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-23T21:17:13.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-23T21:17:13.739 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-23T21:17:14.118 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-23T21:17:14.118 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:14.118 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-23T21:17:14.118 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:17:14.118 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T21:17:14.118 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T21:17:14.118 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T21:17:14.118 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T21:17:14.118 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T21:17:14.119 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T21:17:14.119 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T21:17:14.119 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T21:17:14.119 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T21:17:14.119 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T21:17:14.119 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T21:17:14.119 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:14.119 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:14.128 INFO:teuthology.orchestra.run.smithi148.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 232 kB/s | 80 kB 00:00 2024-04-23T21:17:14.195 INFO:teuthology.orchestra.run.smithi148.stdout:(2/2): ceph-fuse-19.0.0-3212.g8254cb5a.el9.x86_ 2.1 MB/s | 890 kB 00:00 2024-04-23T21:17:14.195 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:14.196 INFO:teuthology.orchestra.run.smithi148.stdout:Total 1.7 MB/s | 970 kB 00:00 2024-04-23T21:17:14.207 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:17:14.227 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:17:14.228 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:17:14.304 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:17:14.305 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:17:14.484 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:17:14.534 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-rook 2024-04-23T21:17:14.622 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-23T21:17:14.684 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:19.0.0-3212.g8254cb5a.el9.x86_64 2/2 2024-04-23T21:17:15.058 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:25 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:15.178 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:15.180 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:15.180 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-23T21:17:15.180 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:15.180 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-23T21:17:15.180 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 49 k 2024-04-23T21:17:15.180 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-04-23T21:17:15.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T21:17:15.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T21:17:15.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T21:17:15.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T21:17:15.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T21:17:15.180 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T21:17:15.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T21:17:15.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T21:17:15.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T21:17:15.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T21:17:15.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T21:17:15.181 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T21:17:15.181 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:15.181 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:17:15.181 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:15.181 INFO:teuthology.orchestra.run.smithi150.stdout:Install 13 Packages 2024-04-23T21:17:15.181 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:15.182 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 2.2 M 2024-04-23T21:17:15.182 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 26 M 2024-04-23T21:17:15.182 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:17:15.290 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-fuse-2:19.0.0-3212.g8254cb5a.el9.x86_64 2/2 2024-04-23T21:17:15.291 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/2 2024-04-23T21:17:15.468 INFO:teuthology.orchestra.run.smithi150.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 250 kB/s | 19 kB 00:00 2024-04-23T21:17:15.494 INFO:teuthology.orchestra.run.smithi150.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 251 kB/s | 26 kB 00:00 2024-04-23T21:17:15.534 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-23T21:17:15.534 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:15.535 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T21:17:15.535 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:19.0.0-3212.g8254cb5a.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-23T21:17:15.535 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:15.535 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:15.577 INFO:teuthology.orchestra.run.smithi150.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-04-23T21:17:15.603 INFO:teuthology.orchestra.run.smithi150.stdout:(4/13): ceph-mgr-rook-19.0.0-3212.g8254cb5a.el9 233 kB/s | 49 kB 00:00 2024-04-23T21:17:15.636 INFO:teuthology.orchestra.run.smithi150.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-04-23T21:17:15.661 INFO:teuthology.orchestra.run.smithi150.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.2 MB/s | 279 kB 00:00 2024-04-23T21:17:15.687 INFO:teuthology.orchestra.run.smithi150.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-04-23T21:17:15.771 INFO:teuthology.orchestra.run.smithi150.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 138 kB/s | 15 kB 00:00 2024-04-23T21:17:15.791 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-volume 2024-04-23T21:17:15.795 INFO:teuthology.orchestra.run.smithi150.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 202 kB/s | 32 kB 00:00 2024-04-23T21:17:15.896 INFO:teuthology.orchestra.run.smithi150.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 590 kB/s | 59 kB 00:00 2024-04-23T21:17:15.921 INFO:teuthology.orchestra.run.smithi150.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 877 kB/s | 206 kB 00:00 2024-04-23T21:17:15.980 INFO:teuthology.orchestra.run.smithi150.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2024-04-23T21:17:16.030 INFO:teuthology.orchestra.run.smithi150.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.9 MB/s | 1.0 MB 00:00 2024-04-23T21:17:16.030 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:16.030 INFO:teuthology.orchestra.run.smithi150.stdout:Total 2.6 MB/s | 2.2 MB 00:00 2024-04-23T21:17:16.092 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:17:16.107 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:17:16.107 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:17:16.190 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:17:16.191 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:17:16.306 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:33 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:16.429 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:16.430 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:16.430 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-04-23T21:17:16.430 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:16.430 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T21:17:16.430 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 273 k 2024-04-23T21:17:16.430 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-04-23T21:17:16.430 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T21:17:16.430 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:16.430 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:17:16.431 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:16.431 INFO:teuthology.orchestra.run.smithi148.stdout:Install 2 Packages 2024-04-23T21:17:16.431 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:16.431 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 350 k 2024-04-23T21:17:16.431 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 1.5 M 2024-04-23T21:17:16.431 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:17:16.688 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:17:16.801 INFO:teuthology.orchestra.run.smithi148.stdout:(1/2): ceph-volume-19.0.0-3212.g8254cb5a.el9.no 1.1 MB/s | 273 kB 00:00 2024-04-23T21:17:16.869 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-23T21:17:16.931 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-23T21:17:16.993 INFO:teuthology.orchestra.run.smithi148.stdout:(2/2): python3-packaging-20.9-5.el9.noarch.rpm 178 kB/s | 77 kB 00:00 2024-04-23T21:17:16.994 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:16.994 INFO:teuthology.orchestra.run.smithi148.stdout:Total 622 kB/s | 350 kB 00:00 2024-04-23T21:17:16.995 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-23T21:17:17.006 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:17:17.015 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:17:17.015 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:17:17.032 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-23T21:17:17.041 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:17:17.041 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:17:17.073 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-23T21:17:17.183 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:17:17.183 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-23T21:17:17.302 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-23T21:17:17.351 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-23T21:17:17.375 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-23T21:17:17.404 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-volume-2:19.0.0-3212.g8254cb5a.el9.noarch 2/2 2024-04-23T21:17:17.419 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-volume-2:19.0.0-3212.g8254cb5a.el9.noarch 2/2 2024-04-23T21:17:17.419 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T21:17:17.419 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T21:17:17.419 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:18.066 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-23T21:17:18.102 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-volume-2:19.0.0-3212.g8254cb5a.el9.noarch 1/2 2024-04-23T21:17:18.114 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-23T21:17:18.148 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-23T21:17:18.198 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-23T21:17:18.226 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:19.0.0-3212.g8254cb5a.el9.noarch 13/13 2024-04-23T21:17:18.365 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-23T21:17:18.365 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:18.365 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T21:17:18.365 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:17:18.365 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T21:17:18.366 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:18.366 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:18.673 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librados-devel 2024-04-23T21:17:18.987 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3212.g8254cb5a.el9.noarch 13/13 2024-04-23T21:17:18.987 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3212.g8254cb5a.el9.noarch 1/13 2024-04-23T21:17:18.988 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-23T21:17:18.988 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-23T21:17:18.988 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-23T21:17:18.988 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-23T21:17:18.988 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-23T21:17:18.988 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-23T21:17:18.988 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-23T21:17:18.988 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-23T21:17:18.988 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-23T21:17:18.988 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-23T21:17:18.988 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-23T21:17:19.194 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:36 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:19.295 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-23T21:17:19.295 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:19.295 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-23T21:17:19.295 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:17:19.295 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T21:17:19.295 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T21:17:19.295 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T21:17:19.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T21:17:19.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T21:17:19.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T21:17:19.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T21:17:19.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T21:17:19.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T21:17:19.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T21:17:19.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T21:17:19.296 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T21:17:19.296 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:19.296 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:19.321 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:19.322 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:19.322 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-04-23T21:17:19.322 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:19.322 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T21:17:19.322 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 126 k 2024-04-23T21:17:19.322 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:19.322 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:17:19.323 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:19.323 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-04-23T21:17:19.323 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:19.323 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 126 k 2024-04-23T21:17:19.323 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 456 k 2024-04-23T21:17:19.323 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:17:19.545 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-cephadm 2024-04-23T21:17:19.641 INFO:teuthology.orchestra.run.smithi148.stdout:librados-devel-19.0.0-3212.g8254cb5a.el9.x86_64 398 kB/s | 126 kB 00:00 2024-04-23T21:17:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:Total 396 kB/s | 126 kB 00:00 2024-04-23T21:17:19.642 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:17:19.648 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:17:19.648 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:17:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:17:19.711 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:17:19.857 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:17:19.954 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:20.050 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:30 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:20.170 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:20.171 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:20.171 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-23T21:17:20.171 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:20.171 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-23T21:17:20.171 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 142 k 2024-04-23T21:17:20.171 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-04-23T21:17:20.171 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T21:17:20.172 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T21:17:20.172 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T21:17:20.172 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T21:17:20.172 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T21:17:20.172 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:20.172 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:17:20.172 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:20.172 INFO:teuthology.orchestra.run.smithi150.stdout:Install 6 Packages 2024-04-23T21:17:20.172 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:20.173 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 7.0 M 2024-04-23T21:17:20.173 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 33 M 2024-04-23T21:17:20.173 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:17:20.375 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados-devel-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:20.646 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:20.646 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:20.646 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T21:17:20.646 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:17:20.646 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:20.646 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:20.876 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install libcephfs2 2024-04-23T21:17:21.149 INFO:teuthology.orchestra.run.smithi150.stdout:(1/6): ceph-mgr-cephadm-19.0.0-3212.g8254cb5a.e 407 kB/s | 142 kB 00:00 2024-04-23T21:17:21.393 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:38 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:21.474 INFO:teuthology.orchestra.run.smithi148.stdout:Package libcephfs2-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:17:21.516 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:21.517 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-04-23T21:17:21.517 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:21.582 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install libcephfs-devel 2024-04-23T21:17:21.824 INFO:teuthology.orchestra.run.smithi150.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 242 kB/s | 248 kB 00:01 2024-04-23T21:17:21.849 INFO:teuthology.orchestra.run.smithi150.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 785 kB/s | 548 kB 00:00 2024-04-23T21:17:22.033 INFO:teuthology.orchestra.run.smithi150.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 278 kB/s | 51 kB 00:00 2024-04-23T21:17:22.096 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:39 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:22.215 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:22.216 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:22.216 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-04-23T21:17:22.216 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:22.216 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T21:17:22.216 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 32 k 2024-04-23T21:17:22.216 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:22.216 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:17:22.216 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:22.216 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-04-23T21:17:22.216 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:22.216 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 32 k 2024-04-23T21:17:22.216 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 142 k 2024-04-23T21:17:22.217 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:17:22.241 INFO:teuthology.orchestra.run.smithi150.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 138 kB/s | 58 kB 00:00 2024-04-23T21:17:22.461 INFO:teuthology.orchestra.run.smithi148.stdout:libcephfs-devel-19.0.0-3212.g8254cb5a.el9.x86_6 130 kB/s | 32 kB 00:00 2024-04-23T21:17:22.461 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:22.462 INFO:teuthology.orchestra.run.smithi148.stdout:Total 129 kB/s | 32 kB 00:00 2024-04-23T21:17:22.462 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:17:22.466 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:17:22.466 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:17:22.468 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:17:22.468 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:17:22.556 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:17:22.667 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:23.028 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:23.246 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:23.247 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:23.247 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T21:17:23.247 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:17:23.247 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:23.247 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:23.434 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librados2 2024-04-23T21:17:23.948 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:40 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:24.030 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:17:24.073 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:24.074 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-04-23T21:17:24.074 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:24.150 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install librbd1 2024-04-23T21:17:24.679 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:41 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:24.758 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:17:24.801 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:24.802 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-04-23T21:17:24.802 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:24.863 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rados 2024-04-23T21:17:25.376 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:42 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:25.455 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rados-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:17:25.496 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:25.497 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-04-23T21:17:25.497 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:25.559 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rgw 2024-04-23T21:17:26.067 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:43 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:26.146 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rgw-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:17:26.186 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:26.187 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-04-23T21:17:26.187 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:26.256 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-cephfs 2024-04-23T21:17:26.765 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:43 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:26.843 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-cephfs-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:17:26.885 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:26.886 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-04-23T21:17:26.886 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:26.953 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install python3-rbd 2024-04-23T21:17:27.464 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:44 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:27.542 INFO:teuthology.orchestra.run.smithi148.stdout:Package python3-rbd-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:17:27.584 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:27.585 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-04-23T21:17:27.585 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:27.644 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-fuse 2024-04-23T21:17:28.151 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:45 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:28.269 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:28.270 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:28.270 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-04-23T21:17:28.270 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:28.270 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T21:17:28.271 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 85 k 2024-04-23T21:17:28.271 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:28.271 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:17:28.271 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:28.271 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-04-23T21:17:28.271 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:28.271 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 85 k 2024-04-23T21:17:28.271 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 231 k 2024-04-23T21:17:28.271 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:17:28.589 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-fuse-19.0.0-3212.g8254cb5a.el9.x86_64.rpm 268 kB/s | 85 kB 00:00 2024-04-23T21:17:28.590 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:28.590 INFO:teuthology.orchestra.run.smithi148.stdout:Total 267 kB/s | 85 kB 00:00 2024-04-23T21:17:28.591 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:17:28.597 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:17:28.598 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:17:28.659 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:17:28.660 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:17:28.774 INFO:teuthology.orchestra.run.smithi150.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 766 kB/s | 6.0 MB 00:07 2024-04-23T21:17:28.776 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:28.776 INFO:teuthology.orchestra.run.smithi150.stdout:Total 831 kB/s | 7.0 MB 00:08 2024-04-23T21:17:28.799 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:17:28.848 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:17:28.860 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:17:28.860 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:17:28.893 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:28.956 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:17:28.956 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:17:29.267 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:17:29.275 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-fuse-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:29.425 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-23T21:17:29.476 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-23T21:17:29.498 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:29.498 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:29.499 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T21:17:29.499 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:17:29.499 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:29.499 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:29.759 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-mirror 2024-04-23T21:17:29.987 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-23T21:17:30.070 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-23T21:17:30.133 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-23T21:17:30.163 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 6/6 2024-04-23T21:17:30.286 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:47 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:30.408 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:30.409 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:30.409 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repo Size 2024-04-23T21:17:30.409 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:30.409 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T21:17:30.410 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 3.1 M 2024-04-23T21:17:30.410 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:30.410 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:17:30.410 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:30.410 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-04-23T21:17:30.410 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:30.410 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 3.1 M 2024-04-23T21:17:30.410 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 13 M 2024-04-23T21:17:30.410 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:17:30.872 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 6/6 2024-04-23T21:17:30.872 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 1/6 2024-04-23T21:17:30.872 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-23T21:17:30.872 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-23T21:17:30.872 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-23T21:17:30.873 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-23T21:17:31.116 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-23T21:17:31.116 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:31.117 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-23T21:17:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:17:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T21:17:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T21:17:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T21:17:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T21:17:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T21:17:31.117 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:31.117 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:31.346 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-fuse 2024-04-23T21:17:31.858 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:41 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:31.978 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:31.979 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-23T21:17:31.979 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:31.979 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-23T21:17:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 890 k 2024-04-23T21:17:31.979 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-04-23T21:17:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T21:17:31.979 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:31.979 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:17:31.979 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:31.980 INFO:teuthology.orchestra.run.smithi150.stdout:Install 2 Packages 2024-04-23T21:17:31.980 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:31.980 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 970 k 2024-04-23T21:17:31.980 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 2.8 M 2024-04-23T21:17:31.980 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:17:32.017 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-mirror-19.0.0-3212.g8254cb5a.el9.x86_64.rpm 1.9 MB/s | 3.1 MB 00:01 2024-04-23T21:17:32.018 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:32.019 INFO:teuthology.orchestra.run.smithi148.stdout:Total 1.9 MB/s | 3.1 MB 00:01 2024-04-23T21:17:32.019 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:17:32.030 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:17:32.031 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:17:32.114 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:17:32.114 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:17:32.368 INFO:teuthology.orchestra.run.smithi150.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 741 kB/s | 80 kB 00:00 2024-04-23T21:17:32.544 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:17:32.610 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:32.635 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:32.635 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T21:17:32.635 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T21:17:32.635 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T21:17:32.635 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T21:17:32.635 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:32.802 INFO:teuthology.orchestra.run.smithi150.stdout:(2/2): ceph-fuse-19.0.0-3212.g8254cb5a.el9.x86_ 1.6 MB/s | 890 kB 00:00 2024-04-23T21:17:32.802 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:32.802 INFO:teuthology.orchestra.run.smithi150.stdout:Total 1.2 MB/s | 970 kB 00:00 2024-04-23T21:17:32.814 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:17:32.836 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:17:32.837 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:17:32.915 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:17:32.916 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:17:33.097 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:17:33.237 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-23T21:17:33.316 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:19.0.0-3212.g8254cb5a.el9.x86_64 2/2 2024-04-23T21:17:33.631 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:33.632 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:33.632 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T21:17:33.632 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:17:33.632 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:33.632 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:33.877 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-nbd 2024-04-23T21:17:34.011 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-fuse-2:19.0.0-3212.g8254cb5a.el9.x86_64 2/2 2024-04-23T21:17:34.011 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/2 2024-04-23T21:17:34.286 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-23T21:17:34.286 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:34.287 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-23T21:17:34.287 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:19.0.0-3212.g8254cb5a.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-23T21:17:34.287 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:34.287 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:34.400 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:51 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:34.528 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:34.530 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:34.530 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-04-23T21:17:34.530 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:34.530 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-04-23T21:17:34.530 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 170 k 2024-04-23T21:17:34.531 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:34.531 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-04-23T21:17:34.531 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-04-23T21:17:34.531 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-04-23T21:17:34.531 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:34.531 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 170 k 2024-04-23T21:17:34.532 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 494 k 2024-04-23T21:17:34.532 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-04-23T21:17:34.567 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-volume 2024-04-23T21:17:34.896 INFO:teuthology.orchestra.run.smithi148.stdout:rbd-nbd-19.0.0-3212.g8254cb5a.el9.x86_64.rpm 466 kB/s | 170 kB 00:00 2024-04-23T21:17:34.897 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:34.897 INFO:teuthology.orchestra.run.smithi148.stdout:Total 464 kB/s | 170 kB 00:00 2024-04-23T21:17:34.897 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-04-23T21:17:34.905 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-04-23T21:17:34.905 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-04-23T21:17:34.969 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-04-23T21:17:34.969 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-04-23T21:17:35.096 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:45 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:35.140 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-04-23T21:17:35.224 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:35.224 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-23T21:17:35.225 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:35.225 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-23T21:17:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:19.0.0-3212.g8254cb5a.el9 ceph-noarch 273 k 2024-04-23T21:17:35.225 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-04-23T21:17:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T21:17:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:35.225 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:17:35.225 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:35.225 INFO:teuthology.orchestra.run.smithi150.stdout:Install 2 Packages 2024-04-23T21:17:35.225 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:35.226 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:35.226 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 350 k 2024-04-23T21:17:35.226 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 1.5 M 2024-04-23T21:17:35.226 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:17:35.476 INFO:teuthology.orchestra.run.smithi150.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 723 kB/s | 77 kB 00:00 2024-04-23T21:17:35.575 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-nbd-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:35.751 INFO:teuthology.orchestra.run.smithi150.stdout:(2/2): ceph-volume-19.0.0-3212.g8254cb5a.el9.no 714 kB/s | 273 kB 00:00 2024-04-23T21:17:35.752 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:35.752 INFO:teuthology.orchestra.run.smithi150.stdout:Total 665 kB/s | 350 kB 00:00 2024-04-23T21:17:35.763 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:17:35.773 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:17:35.773 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:17:35.799 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:17:35.800 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:17:35.840 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:35.841 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:35.841 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-04-23T21:17:35.841 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:17:35.841 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:35.841 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:35.940 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:17:36.055 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install rbd-mirror 2024-04-23T21:17:36.124 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-23T21:17:36.162 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:19.0.0-3212.g8254cb5a.el9.noarch 2/2 2024-04-23T21:17:36.179 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:19.0.0-3212.g8254cb5a.el9.noarch 2/2 2024-04-23T21:17:36.179 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T21:17:36.179 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T21:17:36.179 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:36.586 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:01:53 ago on Tue 23 Apr 2024 09:15:43 PM UTC. 2024-04-23T21:17:36.668 INFO:teuthology.orchestra.run.smithi148.stdout:Package rbd-mirror-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:17:36.709 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-04-23T21:17:36.710 INFO:teuthology.orchestra.run.smithi148.stdout:Nothing to do. 2024-04-23T21:17:36.710 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-04-23T21:17:36.798 DEBUG:teuthology.parallel:result is None 2024-04-23T21:17:36.897 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:19.0.0-3212.g8254cb5a.el9.noarch 1/2 2024-04-23T21:17:37.217 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-23T21:17:37.217 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:37.217 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-23T21:17:37.217 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:19.0.0-3212.g8254cb5a.el9.noarch 2024-04-23T21:17:37.217 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T21:17:37.217 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:37.217 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:37.534 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install librados-devel 2024-04-23T21:17:38.047 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:48 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:38.171 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:38.172 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-04-23T21:17:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-23T21:17:38.172 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 126 k 2024-04-23T21:17:38.172 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:17:38.172 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-04-23T21:17:38.173 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 126 k 2024-04-23T21:17:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 456 k 2024-04-23T21:17:38.173 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:17:38.491 INFO:teuthology.orchestra.run.smithi150.stdout:librados-devel-19.0.0-3212.g8254cb5a.el9.x86_64 398 kB/s | 126 kB 00:00 2024-04-23T21:17:38.492 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:38.492 INFO:teuthology.orchestra.run.smithi150.stdout:Total 396 kB/s | 126 kB 00:00 2024-04-23T21:17:38.492 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:17:38.497 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:17:38.498 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:17:38.560 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:17:38.561 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:17:38.714 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:17:38.812 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:39.214 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados-devel-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:39.449 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:39.449 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:39.449 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-23T21:17:39.449 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:17:39.449 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:39.449 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:39.665 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install libcephfs2 2024-04-23T21:17:40.186 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:50 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:40.266 INFO:teuthology.orchestra.run.smithi150.stdout:Package libcephfs2-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:17:40.307 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:40.309 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-04-23T21:17:40.309 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:40.372 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install libcephfs-devel 2024-04-23T21:17:40.888 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:50 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:41.009 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:41.010 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:41.010 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-04-23T21:17:41.010 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:41.011 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-23T21:17:41.011 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 32 k 2024-04-23T21:17:41.011 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:41.011 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:17:41.011 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:41.011 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-04-23T21:17:41.012 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:41.012 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 32 k 2024-04-23T21:17:41.012 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 142 k 2024-04-23T21:17:41.012 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:17:41.186 INFO:teuthology.orchestra.run.smithi150.stdout:libcephfs-devel-19.0.0-3212.g8254cb5a.el9.x86_6 180 kB/s | 32 kB 00:00 2024-04-23T21:17:41.187 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:41.187 INFO:teuthology.orchestra.run.smithi150.stdout:Total 179 kB/s | 32 kB 00:00 2024-04-23T21:17:41.187 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:17:41.191 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:17:41.191 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:17:41.193 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:17:41.193 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:17:41.278 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:17:41.398 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:41.751 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:41.948 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:41.949 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:41.949 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-23T21:17:41.949 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:17:41.949 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:41.949 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:42.101 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install librados2 2024-04-23T21:17:42.613 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:52 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:42.692 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:17:42.733 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:42.734 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-04-23T21:17:42.734 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:42.795 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install librbd1 2024-04-23T21:17:43.312 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:53 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:43.391 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:17:43.432 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:43.433 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-04-23T21:17:43.433 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:43.488 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-rados 2024-04-23T21:17:43.998 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:53 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:44.077 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-rados-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:17:44.118 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:44.119 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-04-23T21:17:44.119 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:44.174 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-rgw 2024-04-23T21:17:44.690 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:54 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:44.770 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-rgw-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:17:44.811 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:44.812 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-04-23T21:17:44.812 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:44.880 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-cephfs 2024-04-23T21:17:45.396 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:55 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:45.475 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-cephfs-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:17:45.516 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:45.517 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-04-23T21:17:45.517 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:45.578 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-rbd 2024-04-23T21:17:46.093 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:56 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:46.173 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-rbd-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:17:46.214 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:46.215 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-04-23T21:17:46.215 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:46.287 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install rbd-fuse 2024-04-23T21:17:46.794 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:56 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:46.913 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:46.914 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:46.914 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-04-23T21:17:46.914 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:46.914 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-23T21:17:46.914 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 85 k 2024-04-23T21:17:46.914 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:46.915 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:17:46.915 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:46.915 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-04-23T21:17:46.915 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:46.915 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 85 k 2024-04-23T21:17:46.915 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 231 k 2024-04-23T21:17:46.915 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:17:47.473 INFO:teuthology.orchestra.run.smithi150.stdout:rbd-fuse-19.0.0-3212.g8254cb5a.el9.x86_64.rpm 153 kB/s | 85 kB 00:00 2024-04-23T21:17:47.473 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:47.474 INFO:teuthology.orchestra.run.smithi150.stdout:Total 152 kB/s | 85 kB 00:00 2024-04-23T21:17:47.474 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:17:47.481 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:17:47.481 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:17:47.544 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:17:47.545 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:17:47.698 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:17:47.793 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:48.119 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-fuse-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:48.333 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:48.334 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:48.334 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-23T21:17:48.334 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:17:48.334 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:48.334 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:48.535 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install rbd-mirror 2024-04-23T21:17:49.055 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:59 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:49.176 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:49.177 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:49.177 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-04-23T21:17:49.177 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:49.177 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-23T21:17:49.177 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 3.1 M 2024-04-23T21:17:49.177 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:49.177 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:17:49.177 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:49.177 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-04-23T21:17:49.177 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:49.177 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 3.1 M 2024-04-23T21:17:49.177 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 13 M 2024-04-23T21:17:49.177 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:17:49.768 INFO:teuthology.orchestra.run.smithi150.stdout:rbd-mirror-19.0.0-3212.g8254cb5a.el9.x86_64.rpm 5.2 MB/s | 3.1 MB 00:00 2024-04-23T21:17:49.768 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:49.769 INFO:teuthology.orchestra.run.smithi150.stdout:Total 5.2 MB/s | 3.1 MB 00:00 2024-04-23T21:17:49.769 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:17:49.778 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:17:49.778 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:17:49.860 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:17:49.861 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:17:50.289 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:17:50.352 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:50.377 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:50.377 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T21:17:50.377 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T21:17:50.377 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T21:17:50.377 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T21:17:50.377 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:51.456 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-23T21:17:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:17:51.456 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:51.456 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:51.671 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install rbd-nbd 2024-04-23T21:17:52.190 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:02:02 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:52.315 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:52.315 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:52.316 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-04-23T21:17:52.316 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:52.316 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-23T21:17:52.316 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:19.0.0-3212.g8254cb5a.el9 ceph 170 k 2024-04-23T21:17:52.316 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:52.316 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-23T21:17:52.316 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-04-23T21:17:52.316 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-04-23T21:17:52.316 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:52.316 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 170 k 2024-04-23T21:17:52.316 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 494 k 2024-04-23T21:17:52.316 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-23T21:17:52.664 INFO:teuthology.orchestra.run.smithi150.stdout:rbd-nbd-19.0.0-3212.g8254cb5a.el9.x86_64.rpm 491 kB/s | 170 kB 00:00 2024-04-23T21:17:52.665 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-23T21:17:52.665 INFO:teuthology.orchestra.run.smithi150.stdout:Total 488 kB/s | 170 kB 00:00 2024-04-23T21:17:52.665 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-23T21:17:52.673 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-23T21:17:52.673 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-23T21:17:52.736 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-23T21:17:52.737 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-23T21:17:52.890 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-23T21:17:52.995 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:53.388 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-nbd-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:19.0.0-3212.g8254cb5a.el9.x86_64 1/1 2024-04-23T21:17:53.639 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:53.639 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-23T21:17:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:19.0.0-3212.g8254cb5a.el9.x86_64 2024-04-23T21:17:53.640 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:53.640 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:53.856 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install rbd-mirror 2024-04-23T21:17:54.382 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:02:04 ago on Tue 23 Apr 2024 09:15:50 PM UTC. 2024-04-23T21:17:54.463 INFO:teuthology.orchestra.run.smithi150.stdout:Package rbd-mirror-2:19.0.0-3212.g8254cb5a.el9.x86_64 is already installed. 2024-04-23T21:17:54.505 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-23T21:17:54.506 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-04-23T21:17:54.506 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-23T21:17:54.569 DEBUG:teuthology.parallel:result is None 2024-04-23T21:17:54.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8254cb5a247caa8e3d35e534a1fefe2362a0a4b8 2024-04-23T21:17:54.717 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T21:17:54.741 INFO:teuthology.orchestra.run.smithi148.stdout:19.0.0-3212.g8254cb5a.el9 2024-04-23T21:17:54.742 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3212.g8254cb5a.el9 2024-04-23T21:17:54.742 INFO:teuthology.task.install:The correct ceph version 19.0.0-3212.g8254cb5a is installed. 2024-04-23T21:17:54.744 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8254cb5a247caa8e3d35e534a1fefe2362a0a4b8 2024-04-23T21:17:54.953 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T21:17:54.974 INFO:teuthology.orchestra.run.smithi150.stdout:19.0.0-3212.g8254cb5a.el9 2024-04-23T21:17:54.974 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3212.g8254cb5a.el9 2024-04-23T21:17:54.974 INFO:teuthology.task.install:The correct ceph version 19.0.0-3212.g8254cb5a is installed. 2024-04-23T21:17:54.976 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-23T21:17:54.976 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:17:54.976 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T21:17:55.005 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:17:55.005 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T21:17:55.038 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-23T21:17:55.039 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:17:55.039 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T21:17:55.071 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T21:17:55.137 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:17:55.137 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T21:17:55.161 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T21:17:55.225 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-23T21:17:55.226 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:17:55.226 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T21:17:55.252 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T21:17:55.317 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:17:55.317 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T21:17:55.340 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T21:17:55.404 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-23T21:17:55.404 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:17:55.404 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T21:17:55.429 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T21:17:55.493 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:17:55.493 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T21:17:55.516 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T21:17:55.579 INFO:teuthology.run_tasks:Running task ceph... 2024-04-23T21:17:55.659 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-23T21:17:55.659 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /var/log/ceph 2024-04-23T21:17:55.661 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /var/log/ceph 2024-04-23T21:17:55.699 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-23T21:17:55.699 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-23T21:17:55.742 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-23T21:17:55.765 INFO:tasks.ceph:Creating extra log directories... 2024-04-23T21:17:55.765 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-23T21:17:55.806 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-23T21:17:55.843 INFO:tasks.ceph:Creating ceph cluster cluster1... 2024-04-23T21:17:55.843 INFO:tasks.ceph:config {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'cluster1', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-23T21:17:55.843 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi/7670210', 'branch': 'main', 'description': 'rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_latest} workloads/rbd-mirror-journal-stress-workunit}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7670210', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'run two ceph clusters and install rbd-mirror'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': '2 ceph clusters with 1 mon and 3 osds each'}, {'desc': 'run four rbd-mirror daemons per cluster'}, {'desc': 'run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon'}], 'name': 'teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'sha1': '8254cb5a247caa8e3d35e534a1fefe2362a0a4b8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8254cb5a247caa8e3d35e534a1fefe2362a0a4b8'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 950, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'], ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']], 'seed': 4807, 'sha1': '8254cb5a247caa8e3d35e534a1fefe2362a0a4b8', 'sleep_before_teardown': 0, 'subset': '19/128', 'suite': 'rbd', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f', 'targets': {'smithi148.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMX5JWB1fO38Pml0tJfiB3TVJ7QYRvuufJe0BcdYHxgyd5yTnCGYAITfnpWXKHm3rywGVVbfw5bTG0hTY1OpztA=', 'smithi150.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBS9ZCMITkD7l57bj+JwW3X3sW3qR6Mz57CYuRA4xv0lgLTn9nZ1ZGAs6/+IyWptTbmmMLbQ/Re0zEwOe4t0Brk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': ['rbd-mirror'], 'flavor': 'default', 'sha1': '8254cb5a247caa8e3d35e534a1fefe2362a0a4b8'}}, {'ceph': {'cluster': 'cluster1', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'sha1': '8254cb5a247caa8e3d35e534a1fefe2362a0a4b8'}}, {'ceph': {'cluster': 'cluster2'}}, {'exec': {'cluster1.client.mirror.0': ["sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'"]}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.3', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.3', 'thrash': True}}, {'rbd-mirror-thrash': {'cluster': 'cluster1'}}, {'rbd-mirror-thrash': {'cluster': 'cluster2'}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/rbd_mirror_stress.sh']}, 'env': {'CEPH_ARGS': '', 'MIRROR_POOL_MODE': 'pool', 'RBD_MIRROR_INSTANCES': '4', 'RBD_MIRROR_MODE': 'journal', 'RBD_MIRROR_USE_EXISTING_CLUSTER': '1', 'RBD_MIRROR_USE_RBD_MIRROR': '1'}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-23_20:16:13', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-23T21:17:55.844 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2024-04-23T21:17:55.873 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2024-04-23T21:17:55.887 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/run/ceph 2024-04-23T21:17:55.928 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/run/ceph 2024-04-23T21:17:55.951 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:17:55.951 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T21:17:56.006 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T21:17:56.006 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-04-23T21:17:56.061 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T21:17:56.061 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T21:17:56.062 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-23T21:17:56.062 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T21:17:56.062 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T21:17:56.062 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-23 21:17:32.887237407 +0000 2024-04-23T21:17:56.062 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-23 21:15:00.573292726 +0000 2024-04-23T21:17:56.062 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-23 21:15:00.573292726 +0000 2024-04-23T21:17:56.062 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-23 21:15:00.573292726 +0000 2024-04-23T21:17:56.062 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T21:17:56.124 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-23T21:17:56.124 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-23T21:17:56.124 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000214722 s, 2.4 MB/s 2024-04-23T21:17:56.125 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T21:17:56.181 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-04-23T21:17:56.238 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T21:17:56.238 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T21:17:56.238 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-23T21:17:56.238 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T21:17:56.238 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T21:17:56.238 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-23 21:17:32.887237407 +0000 2024-04-23T21:17:56.238 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-23 21:15:00.916284459 +0000 2024-04-23T21:17:56.238 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-23 21:15:00.916284459 +0000 2024-04-23T21:17:56.238 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-23 21:15:00.916284459 +0000 2024-04-23T21:17:56.239 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T21:17:56.300 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-23T21:17:56.300 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-23T21:17:56.300 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000184007 s, 2.8 MB/s 2024-04-23T21:17:56.302 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T21:17:56.358 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-04-23T21:17:56.413 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T21:17:56.413 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T21:17:56.413 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-23T21:17:56.414 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T21:17:56.414 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T21:17:56.414 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-23 21:17:32.888237384 +0000 2024-04-23T21:17:56.414 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-23 21:15:01.243276579 +0000 2024-04-23T21:17:56.414 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-23 21:15:01.243276579 +0000 2024-04-23T21:17:56.414 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-23 21:15:01.243276579 +0000 2024-04-23T21:17:56.414 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T21:17:56.475 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-23T21:17:56.476 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-23T21:17:56.476 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000200248 s, 2.6 MB/s 2024-04-23T21:17:56.477 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T21:17:56.533 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-04-23T21:17:56.588 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T21:17:56.588 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T21:17:56.588 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-23T21:17:56.588 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T21:17:56.588 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T21:17:56.589 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-04-23 21:17:32.888237384 +0000 2024-04-23T21:17:56.589 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-04-23 21:15:01.575268577 +0000 2024-04-23T21:17:56.589 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-04-23 21:15:01.575268577 +0000 2024-04-23T21:17:56.589 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-04-23 21:15:01.575268577 +0000 2024-04-23T21:17:56.589 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T21:17:56.650 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-04-23T21:17:56.650 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-04-23T21:17:56.650 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000183497 s, 2.8 MB/s 2024-04-23T21:17:56.652 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T21:17:56.707 INFO:tasks.ceph:osd dev map: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2024-04-23T21:17:56.707 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}} 2024-04-23T21:17:56.707 INFO:tasks.ceph:Generating config... 2024-04-23T21:17:56.708 INFO:tasks.ceph:[client] debug rbd = 20 2024-04-23T21:17:56.708 INFO:tasks.ceph:[client] debug rbd_mirror = 15 2024-04-23T21:17:56.708 INFO:tasks.ceph:[client] log to stderr = False 2024-04-23T21:17:56.708 INFO:tasks.ceph:[client] rbd default features = 125 2024-04-23T21:17:56.708 INFO:tasks.ceph:[client] rbd mirror image policy type = simple 2024-04-23T21:17:56.709 INFO:tasks.ceph:[client.mirror.0] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:17:56.709 INFO:tasks.ceph:[client.mirror.0] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:17:56.709 INFO:tasks.ceph:[client.mirror.1] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:17:56.709 INFO:tasks.ceph:[client.mirror.1] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:17:56.709 INFO:tasks.ceph:[client.mirror.2] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:17:56.709 INFO:tasks.ceph:[client.mirror.2] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:17:56.709 INFO:tasks.ceph:[client.mirror.3] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:17:56.709 INFO:tasks.ceph:[client.mirror.3] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:17:56.709 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-04-23T21:17:56.709 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-04-23T21:17:56.709 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-04-23T21:17:56.709 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-23T21:17:56.709 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-23T21:17:56.709 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-23T21:17:56.709 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-23T21:17:56.710 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-23T21:17:56.710 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-23T21:17:56.710 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-04-23T21:17:56.710 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-04-23T21:17:56.710 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-23T21:17:56.710 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-23T21:17:56.710 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-23T21:17:56.710 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-23T21:17:56.710 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-23T21:17:56.710 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-23T21:17:56.710 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-23T21:17:56.710 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-23T21:17:56.710 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-23T21:17:56.710 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-23T21:17:56.711 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-23T21:17:56.711 INFO:tasks.ceph:Setting up cluster1.mon.a... 2024-04-23T21:17:56.711 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster1.keyring 2024-04-23T21:17:56.822 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/cluster1.keyring 2024-04-23T21:17:56.824 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster1.keyring 2024-04-23T21:17:56.864 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2024-04-23T21:17:56.928 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster1.mon.a', '172.21.15.148')] 2024-04-23T21:17:56.928 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.148', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'mon.a': {}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}} 2024-04-23T21:17:56.929 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:17:56.929 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-23T21:17:56.983 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.148 --print /home/ubuntu/cephtest/cluster1.monmap 2024-04-23T21:17:57.062 INFO:teuthology.orchestra.run.smithi148.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster1.monmap 2024-04-23T21:17:57.062 INFO:teuthology.orchestra.run.smithi148.stdout:monmaptool: generated fsid ed086417-ad7c-481e-9d20-547ac50d08e6 2024-04-23T21:17:57.062 INFO:teuthology.orchestra.run.smithi148.stdout:setting min_mon_release = quincy 2024-04-23T21:17:57.062 INFO:teuthology.orchestra.run.smithi148.stdout:epoch 0 2024-04-23T21:17:57.062 INFO:teuthology.orchestra.run.smithi148.stdout:fsid ed086417-ad7c-481e-9d20-547ac50d08e6 2024-04-23T21:17:57.062 INFO:teuthology.orchestra.run.smithi148.stdout:last_changed 2024-04-23T21:17:57.063543+0000 2024-04-23T21:17:57.062 INFO:teuthology.orchestra.run.smithi148.stdout:created 2024-04-23T21:17:57.063543+0000 2024-04-23T21:17:57.063 INFO:teuthology.orchestra.run.smithi148.stdout:min_mon_release 17 (quincy) 2024-04-23T21:17:57.063 INFO:teuthology.orchestra.run.smithi148.stdout:election_strategy: 1 2024-04-23T21:17:57.063 INFO:teuthology.orchestra.run.smithi148.stdout:0: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.a 2024-04-23T21:17:57.063 INFO:teuthology.orchestra.run.smithi148.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster1.monmap (1 monitors) 2024-04-23T21:17:57.064 DEBUG:teuthology.orchestra.run.smithi148:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-23T21:17:57.119 INFO:tasks.ceph:Writing /etc/ceph/cluster1.conf for FSID ed086417-ad7c-481e-9d20-547ac50d08e6... 2024-04-23T21:17:57.120 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2024-04-23T21:17:57.162 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2024-04-23T21:17:57.199 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-04-23T21:17:57.199 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-04-23T21:17:57.199 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T21:17:57.199 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-04-23T21:17:57.199 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.199 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-04-23T21:17:57.199 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.200 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-04-23T21:17:57.200 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.200 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-04-23T21:17:57.200 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-04-23T21:17:57.200 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.200 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-04-23T21:17:57.200 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-04-23T21:17:57.200 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.200 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T21:17:57.200 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-04-23T21:17:57.200 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.200 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T21:17:57.200 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T21:17:57.201 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.201 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-04-23T21:17:57.201 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.201 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-04-23T21:17:57.201 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-04-23T21:17:57.201 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-04-23T21:17:57.201 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-04-23T21:17:57.201 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-04-23T21:17:57.201 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-04-23T21:17:57.201 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-04-23T21:17:57.201 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T21:17:57.201 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T21:17:57.201 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-04-23T21:17:57.201 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.202 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T21:17:57.202 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.202 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-04-23T21:17:57.202 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.202 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-04-23T21:17:57.202 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.202 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-04-23T21:17:57.202 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-04-23T21:17:57.202 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-04-23T21:17:57.202 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = 172.21.15.148 2024-04-23T21:17:57.202 INFO:teuthology.orchestra.run.smithi148.stdout: mon client directed command retry = 5 2024-04-23T21:17:57.202 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on pool no app = False 2024-04-23T21:17:57.203 INFO:teuthology.orchestra.run.smithi148.stdout: ms inject socket failures = 5000 2024-04-23T21:17:57.203 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = ed086417-ad7c-481e-9d20-547ac50d08e6 2024-04-23T21:17:57.203 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.203 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-04-23T21:17:57.203 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-04-23T21:17:57.203 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.203 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-04-23T21:17:57.203 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2024-04-23T21:17:57.203 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-04-23T21:17:57.203 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.203 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-04-23T21:17:57.203 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-04-23T21:17:57.203 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.203 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-04-23T21:17:57.204 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-04-23T21:17:57.204 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-04-23T21:17:57.204 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug trim objects = true 2024-04-23T21:17:57.204 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.204 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-04-23T21:17:57.204 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-04-23T21:17:57.204 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.204 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-04-23T21:17:57.204 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.204 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-04-23T21:17:57.204 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.204 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-04-23T21:17:57.204 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.204 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-04-23T21:17:57.205 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-04-23T21:17:57.205 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.205 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-04-23T21:17:57.205 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-04-23T21:17:57.205 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-04-23T21:17:57.205 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore compression algorithm = zstd 2024-04-23T21:17:57.205 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore compression mode = aggressive 2024-04-23T21:17:57.205 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-04-23T21:17:57.205 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 1/20 2024-04-23T21:17:57.205 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 1/20 2024-04-23T21:17:57.205 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-23T21:17:57.205 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-04-23T21:17:57.205 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 4/10 2024-04-23T21:17:57.205 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T21:17:57.205 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-04-23T21:17:57.206 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-04-23T21:17:57.206 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-04-23T21:17:57.206 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-04-23T21:17:57.206 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.206 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-04-23T21:17:57.206 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-23T21:17:57.206 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-04-23T21:17:57.206 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-04-23T21:17:57.206 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-04-23T21:17:57.206 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-04-23T21:17:57.206 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-04-23T21:17:57.206 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-04-23T21:17:57.206 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.207 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-04-23T21:17:57.207 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-23T21:17:57.207 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-04-23T21:17:57.207 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-04-23T21:17:57.207 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-04-23T21:17:57.207 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-04-23T21:17:57.207 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-04-23T21:17:57.207 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-04-23T21:17:57.207 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-04-23T21:17:57.207 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-04-23T21:17:57.207 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-04-23T21:17:57.207 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.207 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T21:17:57.207 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T21:17:57.208 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T21:17:57.208 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.208 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T21:17:57.208 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T21:17:57.208 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T21:17:57.208 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.208 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-04-23T21:17:57.208 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-04-23T21:17:57.208 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.208 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-04-23T21:17:57.208 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:17:57.208 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-04-23T21:17:57.208 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-04-23T21:17:57.208 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-04-23T21:17:57.209 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-04-23T21:17:57.209 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T21:17:57.209 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T21:17:57.209 INFO:teuthology.orchestra.run.smithi148.stdout: debug rbd = 20 2024-04-23T21:17:57.209 INFO:teuthology.orchestra.run.smithi148.stdout: debug rbd_mirror = 15 2024-04-23T21:17:57.209 INFO:teuthology.orchestra.run.smithi148.stdout: log to stderr = False 2024-04-23T21:17:57.209 INFO:teuthology.orchestra.run.smithi148.stdout: rbd default features = 125 2024-04-23T21:17:57.209 INFO:teuthology.orchestra.run.smithi148.stdout: rbd mirror image policy type = simple 2024-04-23T21:17:57.209 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-04-23T21:17:57.209 INFO:teuthology.orchestra.run.smithi148.stdout:[client.mirror.0] 2024-04-23T21:17:57.209 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:17:57.209 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:17:57.209 INFO:teuthology.orchestra.run.smithi148.stdout:[client.mirror.1] 2024-04-23T21:17:57.209 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:17:57.209 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:17:57.210 INFO:teuthology.orchestra.run.smithi148.stdout:[client.mirror.2] 2024-04-23T21:17:57.210 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:17:57.210 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:17:57.210 INFO:teuthology.orchestra.run.smithi148.stdout:[client.mirror.3] 2024-04-23T21:17:57.210 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:17:57.210 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:17:57.210 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2024-04-23T21:17:57.211 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2024-04-23T21:17:57.211 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T21:17:57.211 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2024-04-23T21:17:57.211 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.211 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2024-04-23T21:17:57.211 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.211 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2024-04-23T21:17:57.211 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.211 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2024-04-23T21:17:57.211 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2024-04-23T21:17:57.211 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.212 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2024-04-23T21:17:57.212 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2024-04-23T21:17:57.212 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.212 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T21:17:57.212 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2024-04-23T21:17:57.212 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.212 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T21:17:57.212 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T21:17:57.212 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.212 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2024-04-23T21:17:57.212 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.212 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2024-04-23T21:17:57.212 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2024-04-23T21:17:57.212 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2024-04-23T21:17:57.213 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2024-04-23T21:17:57.213 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2024-04-23T21:17:57.213 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2024-04-23T21:17:57.213 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2024-04-23T21:17:57.213 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T21:17:57.213 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T21:17:57.213 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2024-04-23T21:17:57.213 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.213 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T21:17:57.213 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.213 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2024-04-23T21:17:57.213 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.213 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2024-04-23T21:17:57.213 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.214 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2024-04-23T21:17:57.214 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2024-04-23T21:17:57.214 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2024-04-23T21:17:57.214 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = 172.21.15.148 2024-04-23T21:17:57.214 INFO:teuthology.orchestra.run.smithi150.stdout: mon client directed command retry = 5 2024-04-23T21:17:57.214 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on pool no app = False 2024-04-23T21:17:57.214 INFO:teuthology.orchestra.run.smithi150.stdout: ms inject socket failures = 5000 2024-04-23T21:17:57.214 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = ed086417-ad7c-481e-9d20-547ac50d08e6 2024-04-23T21:17:57.214 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.214 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2024-04-23T21:17:57.214 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2024-04-23T21:17:57.214 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.215 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2024-04-23T21:17:57.215 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2024-04-23T21:17:57.215 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2024-04-23T21:17:57.215 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.215 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2024-04-23T21:17:57.215 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2024-04-23T21:17:57.215 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.215 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2024-04-23T21:17:57.215 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2024-04-23T21:17:57.215 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2024-04-23T21:17:57.215 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug trim objects = true 2024-04-23T21:17:57.215 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.215 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2024-04-23T21:17:57.215 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2024-04-23T21:17:57.216 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.216 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2024-04-23T21:17:57.216 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.216 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2024-04-23T21:17:57.216 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.216 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2024-04-23T21:17:57.216 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.216 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2024-04-23T21:17:57.216 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2024-04-23T21:17:57.216 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.216 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2024-04-23T21:17:57.216 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2024-04-23T21:17:57.217 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2024-04-23T21:17:57.217 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore compression algorithm = zstd 2024-04-23T21:17:57.217 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore compression mode = aggressive 2024-04-23T21:17:57.217 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2024-04-23T21:17:57.217 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 1/20 2024-04-23T21:17:57.217 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 1/20 2024-04-23T21:17:57.217 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-23T21:17:57.217 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2024-04-23T21:17:57.217 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 4/10 2024-04-23T21:17:57.217 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T21:17:57.217 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2024-04-23T21:17:57.217 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2024-04-23T21:17:57.217 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2024-04-23T21:17:57.217 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2024-04-23T21:17:57.218 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.218 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2024-04-23T21:17:57.218 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-23T21:17:57.218 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2024-04-23T21:17:57.218 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-04-23T21:17:57.218 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-04-23T21:17:57.218 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-04-23T21:17:57.218 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-04-23T21:17:57.218 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2024-04-23T21:17:57.218 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.218 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2024-04-23T21:17:57.218 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-23T21:17:57.218 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-04-23T21:17:57.219 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2024-04-23T21:17:57.219 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-04-23T21:17:57.219 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2024-04-23T21:17:57.219 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2024-04-23T21:17:57.219 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-04-23T21:17:57.219 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2024-04-23T21:17:57.219 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2024-04-23T21:17:57.219 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-04-23T21:17:57.219 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.219 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T21:17:57.219 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T21:17:57.219 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T21:17:57.219 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.220 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T21:17:57.220 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T21:17:57.220 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T21:17:57.220 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.220 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2024-04-23T21:17:57.220 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2024-04-23T21:17:57.220 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.220 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2024-04-23T21:17:57.220 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:17:57.220 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2024-04-23T21:17:57.220 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2024-04-23T21:17:57.220 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2024-04-23T21:17:57.220 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2024-04-23T21:17:57.220 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T21:17:57.220 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T21:17:57.220 INFO:teuthology.orchestra.run.smithi150.stdout: debug rbd = 20 2024-04-23T21:17:57.221 INFO:teuthology.orchestra.run.smithi150.stdout: debug rbd_mirror = 15 2024-04-23T21:17:57.221 INFO:teuthology.orchestra.run.smithi150.stdout: log to stderr = False 2024-04-23T21:17:57.221 INFO:teuthology.orchestra.run.smithi150.stdout: rbd default features = 125 2024-04-23T21:17:57.221 INFO:teuthology.orchestra.run.smithi150.stdout: rbd mirror image policy type = simple 2024-04-23T21:17:57.221 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2024-04-23T21:17:57.221 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.0] 2024-04-23T21:17:57.221 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:17:57.221 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:17:57.221 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.1] 2024-04-23T21:17:57.221 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:17:57.221 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:17:57.221 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.2] 2024-04-23T21:17:57.221 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:17:57.221 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:17:57.221 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.3] 2024-04-23T21:17:57.222 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:17:57.222 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:17:57.223 INFO:tasks.ceph:Creating admin key on cluster1.mon.a... 2024-04-23T21:17:57.224 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster1.keyring 2024-04-23T21:17:57.315 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-23T21:17:57.315 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:17:57.316 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/cluster1.keyring of=/dev/stdout 2024-04-23T21:17:57.330 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:17:57.330 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/home/ubuntu/cephtest/cluster1.monmap of=/dev/stdout 2024-04-23T21:17:57.385 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi148.front.sepia.ceph.com 2024-04-23T21:17:57.385 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:17:57.385 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/cluster1.keyring 2024-04-23T21:17:57.385 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2024-04-23T21:17:57.457 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:17:57.457 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/cluster1.monmap 2024-04-23T21:17:57.512 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi150.front.sepia.ceph.com 2024-04-23T21:17:57.512 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:17:57.512 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/cluster1.keyring 2024-04-23T21:17:57.512 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2024-04-23T21:17:57.550 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:17:57.550 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/cluster1.monmap 2024-04-23T21:17:57.607 INFO:tasks.ceph:Setting up mon nodes... 2024-04-23T21:17:57.607 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-23T21:17:57.607 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mgr/cluster1-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster1-x/keyring 2024-04-23T21:17:57.655 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mgr/cluster1-x/keyring 2024-04-23T21:17:57.658 INFO:tasks.ceph:Setting up mds nodes... 2024-04-23T21:17:57.658 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-23T21:17:57.659 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/cluster1.client.0.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.0.keyring 2024-04-23T21:17:57.697 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/cluster1.client.0.keyring 2024-04-23T21:17:57.708 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster1.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.keyring 2024-04-23T21:17:57.759 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster1.client.mirror.keyring 2024-04-23T21:17:57.772 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster1.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.0.keyring 2024-04-23T21:17:57.813 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster1.client.mirror.0.keyring 2024-04-23T21:17:57.825 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.1 /etc/ceph/cluster1.client.mirror.1.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.1.keyring 2024-04-23T21:17:57.905 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster1.client.mirror.1.keyring 2024-04-23T21:17:57.917 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.2 /etc/ceph/cluster1.client.mirror.2.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.2.keyring 2024-04-23T21:17:57.995 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster1.client.mirror.2.keyring 2024-04-23T21:17:58.007 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.3 /etc/ceph/cluster1.client.mirror.3.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.3.keyring 2024-04-23T21:17:58.086 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster1.client.mirror.3.keyring 2024-04-23T21:17:58.098 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.4 /etc/ceph/cluster1.client.mirror.4.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.4.keyring 2024-04-23T21:17:58.176 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster1.client.mirror.4.keyring 2024-04-23T21:17:58.188 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.5 /etc/ceph/cluster1.client.mirror.5.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.5.keyring 2024-04-23T21:17:58.266 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster1.client.mirror.5.keyring 2024-04-23T21:17:58.278 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.6 /etc/ceph/cluster1.client.mirror.6.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.6.keyring 2024-04-23T21:17:58.358 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster1.client.mirror.6.keyring 2024-04-23T21:17:58.370 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-23T21:17:58.370 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}} 2024-04-23T21:17:58.370 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/cluster1-0 2024-04-23T21:17:58.396 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2024-04-23T21:17:58.396 INFO:tasks.ceph:role: cluster1.osd.0 2024-04-23T21:17:58.396 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com 2024-04-23T21:17:58.396 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-23T21:17:58.469 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T21:17:58.469 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T21:17:58.469 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T21:17:58.469 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T21:17:58.469 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T21:17:58.470 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-04-23T21:17:58.470 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T21:17:58.470 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T21:17:58.470 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T21:17:58.470 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T21:17:58.980 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-04-23T21:17:58.983 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-04-23T21:17:58.983 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/cluster1-0 2024-04-23T21:17:59.112 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-0 2024-04-23T21:17:59.145 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/cluster1-1 2024-04-23T21:17:59.210 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2024-04-23T21:17:59.211 INFO:tasks.ceph:role: cluster1.osd.1 2024-04-23T21:17:59.211 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com 2024-04-23T21:17:59.211 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-23T21:17:59.285 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T21:17:59.285 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T21:17:59.285 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T21:17:59.285 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T21:17:59.285 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T21:17:59.285 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-04-23T21:17:59.285 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T21:17:59.285 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T21:17:59.286 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T21:17:59.286 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T21:17:59.741 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-04-23T21:17:59.743 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-04-23T21:17:59.743 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster1-1 2024-04-23T21:17:59.837 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-1 2024-04-23T21:17:59.867 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/osd/cluster1-2 2024-04-23T21:17:59.933 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2024-04-23T21:17:59.933 INFO:tasks.ceph:role: cluster1.osd.2 2024-04-23T21:17:59.933 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com 2024-04-23T21:17:59.933 DEBUG:teuthology.orchestra.run.smithi148:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-23T21:18:00.005 INFO:teuthology.orchestra.run.smithi148.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T21:18:00.006 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T21:18:00.006 INFO:teuthology.orchestra.run.smithi148.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T21:18:00.006 INFO:teuthology.orchestra.run.smithi148.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T21:18:00.006 INFO:teuthology.orchestra.run.smithi148.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T21:18:00.006 INFO:teuthology.orchestra.run.smithi148.stdout: = sunit=0 swidth=0 blks 2024-04-23T21:18:00.006 INFO:teuthology.orchestra.run.smithi148.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T21:18:00.006 INFO:teuthology.orchestra.run.smithi148.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T21:18:00.006 INFO:teuthology.orchestra.run.smithi148.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T21:18:00.006 INFO:teuthology.orchestra.run.smithi148.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T21:18:00.502 INFO:teuthology.orchestra.run.smithi148.stdout:Discarding blocks...Done. 2024-04-23T21:18:00.504 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi148.front.sepia.ceph.com -o noatime 2024-04-23T21:18:00.505 DEBUG:teuthology.orchestra.run.smithi148:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster1-2 2024-04-23T21:18:00.611 DEBUG:teuthology.orchestra.run.smithi148:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-2 2024-04-23T21:18:00.643 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster1.monmap 2024-04-23T21:18:00.739 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:00.739+0000 7f80a4c0e640 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-0/keyring: can't open /var/lib/ceph/osd/cluster1-0/keyring: (2) No such file or directory 2024-04-23T21:18:00.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:00.741+0000 7f80a4c0e640 -1 created new key in keyring /var/lib/ceph/osd/cluster1-0/keyring 2024-04-23T21:18:00.740 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:00.741+0000 7f80a4c0e640 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2024-04-23T21:18:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:00.741+0000 7f80a4c0e640 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2024-04-23T21:18:00.741 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:00.741+0000 7f80a4c0e640 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2024-04-23T21:18:00.747 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:00.748+0000 7f80a4c0e640 -1 bluestore(/var/lib/ceph/osd/cluster1-0) _read_fsid unparsable uuid 2024-04-23T21:18:03.597 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-0 2024-04-23T21:18:03.622 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster1.monmap 2024-04-23T21:18:03.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:03.706+0000 7f744781c640 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-1/keyring: can't open /var/lib/ceph/osd/cluster1-1/keyring: (2) No such file or directory 2024-04-23T21:18:03.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:03.707+0000 7f744781c640 -1 created new key in keyring /var/lib/ceph/osd/cluster1-1/keyring 2024-04-23T21:18:03.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:03.707+0000 7f744781c640 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2024-04-23T21:18:03.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:03.707+0000 7f744781c640 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2024-04-23T21:18:03.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:03.707+0000 7f744781c640 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2024-04-23T21:18:03.707 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:03.707+0000 7f744781c640 -1 bluestore(/var/lib/ceph/osd/cluster1-1) _read_fsid unparsable uuid 2024-04-23T21:18:06.568 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-1 2024-04-23T21:18:06.637 DEBUG:teuthology.orchestra.run.smithi148:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster1.monmap 2024-04-23T21:18:06.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:06.727+0000 7fdcba012640 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-2/keyring: can't open /var/lib/ceph/osd/cluster1-2/keyring: (2) No such file or directory 2024-04-23T21:18:06.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:06.727+0000 7fdcba012640 -1 created new key in keyring /var/lib/ceph/osd/cluster1-2/keyring 2024-04-23T21:18:06.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:06.727+0000 7fdcba012640 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2024-04-23T21:18:06.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:06.727+0000 7fdcba012640 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2024-04-23T21:18:06.727 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:06.727+0000 7fdcba012640 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2024-04-23T21:18:06.728 INFO:teuthology.orchestra.run.smithi148.stderr:2024-04-23T21:18:06.728+0000 7fdcba012640 -1 bluestore(/var/lib/ceph/osd/cluster1-2) _read_fsid unparsable uuid 2024-04-23T21:18:09.579 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-2 2024-04-23T21:18:09.604 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-23T21:18:09.604 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:18:09.604 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mgr/cluster1-x/keyring of=/dev/stdout 2024-04-23T21:18:09.669 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:18:09.669 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/cluster1-0/keyring of=/dev/stdout 2024-04-23T21:18:09.734 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:18:09.734 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/cluster1-1/keyring of=/dev/stdout 2024-04-23T21:18:09.797 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:18:09.797 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/cluster1-2/keyring of=/dev/stdout 2024-04-23T21:18:09.861 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:18:09.861 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/cluster1.client.0.keyring of=/dev/stdout 2024-04-23T21:18:09.915 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:09.915 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster1.client.mirror.keyring of=/dev/stdout 2024-04-23T21:18:09.934 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:09.934 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster1.client.mirror.0.keyring of=/dev/stdout 2024-04-23T21:18:09.991 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:09.992 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster1.client.mirror.1.keyring of=/dev/stdout 2024-04-23T21:18:10.048 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:10.049 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster1.client.mirror.2.keyring of=/dev/stdout 2024-04-23T21:18:10.105 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:10.106 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster1.client.mirror.3.keyring of=/dev/stdout 2024-04-23T21:18:10.162 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:10.162 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster1.client.mirror.4.keyring of=/dev/stdout 2024-04-23T21:18:10.218 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:10.218 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster1.client.mirror.5.keyring of=/dev/stdout 2024-04-23T21:18:10.274 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:10.274 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster1.client.mirror.6.keyring of=/dev/stdout 2024-04-23T21:18:10.330 INFO:tasks.ceph:Adding keys to all mons... 2024-04-23T21:18:10.331 DEBUG:teuthology.orchestra.run.smithi148:> sudo tee -a /etc/ceph/cluster1.keyring 2024-04-23T21:18:10.353 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr.x] 2024-04-23T21:18:10.353 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCFJShmbj4xJxAAxmMVD+OkSwBrqe0xrrPdkw== 2024-04-23T21:18:10.354 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.0] 2024-04-23T21:18:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCIJShmc3scLBAA0T9NHZC2yWUeGUE9Fq7R7g== 2024-04-23T21:18:10.354 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.1] 2024-04-23T21:18:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCLJShmg7YxKhAA63fi8LOnYbCHE3cNOHG6HA== 2024-04-23T21:18:10.354 INFO:teuthology.orchestra.run.smithi148.stdout:[osd.2] 2024-04-23T21:18:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCOJShmejdrKxAADp2rQz2+/cGepgrn0mcYxQ== 2024-04-23T21:18:10.354 INFO:teuthology.orchestra.run.smithi148.stdout:[client.0] 2024-04-23T21:18:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCFJShmoOWlKRAARWNhzgQ+9l+fGekLpDaAuA== 2024-04-23T21:18:10.354 INFO:teuthology.orchestra.run.smithi148.stdout:[client.mirror] 2024-04-23T21:18:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCFJShmRR5VLRAAMOVjiRzcJfepgCt6uJB2Xw== 2024-04-23T21:18:10.354 INFO:teuthology.orchestra.run.smithi148.stdout:[client.mirror.0] 2024-04-23T21:18:10.354 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCFJShmJcaNMBAA6LzBW8HDCgEeTNXz25ePzg== 2024-04-23T21:18:10.354 INFO:teuthology.orchestra.run.smithi148.stdout:[client.mirror.1] 2024-04-23T21:18:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCFJShmeUUNNhAAe7juz4n7Jvjj/OkRxPBBnQ== 2024-04-23T21:18:10.355 INFO:teuthology.orchestra.run.smithi148.stdout:[client.mirror.2] 2024-04-23T21:18:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCFJShmGoN1OxAAaYY2nUFrzt1Z+CpAr22zWQ== 2024-04-23T21:18:10.355 INFO:teuthology.orchestra.run.smithi148.stdout:[client.mirror.3] 2024-04-23T21:18:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCGJShm79xEBRAAstBk8oqga/923bSRrLvaog== 2024-04-23T21:18:10.355 INFO:teuthology.orchestra.run.smithi148.stdout:[client.mirror.4] 2024-04-23T21:18:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCGJShmdQCgChAA7H7/FC1JKf/35GylbfD8cA== 2024-04-23T21:18:10.355 INFO:teuthology.orchestra.run.smithi148.stdout:[client.mirror.5] 2024-04-23T21:18:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCGJShmI5/+DxAAaJ403OSw6yCfx2dL2mZwRw== 2024-04-23T21:18:10.355 INFO:teuthology.orchestra.run.smithi148.stdout:[client.mirror.6] 2024-04-23T21:18:10.355 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCGJShmN7B2FRAAtadO7YBs7caJp2bKO2nWVQ== 2024-04-23T21:18:10.356 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-23T21:18:10.446 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T21:18:10.495 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T21:18:10.544 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T21:18:10.592 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:10.643 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:10.694 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:10.745 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:10.797 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:10.848 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:10.900 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.4 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:10.951 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.5 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:11.001 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.6 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:11.052 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-23T21:18:11.052 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mon/cluster1-a 2024-04-23T21:18:11.076 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster1 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster1.monmap --keyring /etc/ceph/cluster1.keyring 2024-04-23T21:18:11.170 DEBUG:teuthology.orchestra.run.smithi148:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster1-a 2024-04-23T21:18:11.194 DEBUG:teuthology.orchestra.run.smithi148:> rm -- /home/ubuntu/cephtest/cluster1.monmap 2024-04-23T21:18:11.249 INFO:tasks.ceph:Starting mon daemons in cluster cluster1... 2024-04-23T21:18:11.250 INFO:tasks.ceph.cluster1.mon.a:Restarting daemon 2024-04-23T21:18:11.250 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster1 -i a 2024-04-23T21:18:11.292 INFO:tasks.ceph.cluster1.mon.a:Started 2024-04-23T21:18:11.292 INFO:tasks.ceph:Starting mgr daemons in cluster cluster1... 2024-04-23T21:18:11.292 INFO:tasks.ceph.cluster1.mgr.x:Restarting daemon 2024-04-23T21:18:11.292 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster1 -i x 2024-04-23T21:18:11.294 INFO:tasks.ceph.cluster1.mgr.x:Started 2024-04-23T21:18:11.294 INFO:tasks.ceph:Setting crush tunables to default 2024-04-23T21:18:11.295 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster cluster1 osd crush tunables default 2024-04-23T21:18:11.467 INFO:teuthology.orchestra.run.smithi148.stderr:adjusted tunables profile to default 2024-04-23T21:18:11.478 INFO:tasks.ceph:check_enable_crimson: False 2024-04-23T21:18:11.478 INFO:tasks.ceph:Starting osd daemons in cluster cluster1... 2024-04-23T21:18:11.478 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:18:11.479 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/cluster1-0/fsid of=/dev/stdout 2024-04-23T21:18:11.502 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:18:11.502 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/cluster1-1/fsid of=/dev/stdout 2024-04-23T21:18:11.566 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:18:11.566 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/osd/cluster1-2/fsid of=/dev/stdout 2024-04-23T21:18:11.629 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster cluster1 osd new c8341e94-a3c7-4414-938d-1eaac8b500b0 0 2024-04-23T21:18:11.673 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:11.673+0000 7ff052638040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T21:18:11.834 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:11.834+0000 7ff052638040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T21:18:11.849 INFO:teuthology.orchestra.run.smithi148.stdout:0 2024-04-23T21:18:11.860 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster cluster1 osd new b9bf3229-a5d5-4ad3-a48f-b5b73d1c32d1 1 2024-04-23T21:18:11.890 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:11.890+0000 7ff052638040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T21:18:12.069 INFO:teuthology.orchestra.run.smithi148.stdout:1 2024-04-23T21:18:12.080 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster cluster1 osd new 1b13468d-227f-433d-88a3-4b93e4862340 2 2024-04-23T21:18:12.169 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:12.169+0000 7ff052638040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T21:18:12.218 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:12.218+0000 7ff052638040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T21:18:12.261 INFO:teuthology.orchestra.run.smithi148.stdout:2 2024-04-23T21:18:12.272 INFO:tasks.ceph.cluster1.osd.0:Restarting daemon 2024-04-23T21:18:12.273 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 0 2024-04-23T21:18:12.275 INFO:tasks.ceph.cluster1.osd.0:Started 2024-04-23T21:18:12.275 INFO:tasks.ceph.cluster1.osd.1:Restarting daemon 2024-04-23T21:18:12.275 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 1 2024-04-23T21:18:12.277 INFO:tasks.ceph.cluster1.osd.1:Started 2024-04-23T21:18:12.277 INFO:tasks.ceph.cluster1.osd.2:Restarting daemon 2024-04-23T21:18:12.277 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 2 2024-04-23T21:18:12.279 INFO:tasks.ceph.cluster1.osd.2:Started 2024-04-23T21:18:12.279 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2024-04-23T21:18:12.466 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:12.466 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":5,"fsid":"ed086417-ad7c-481e-9d20-547ac50d08e6","created":"2024-04-23T21:18:11.375874+0000","modified":"2024-04-23T21:18:12.261963+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T21:18:12.261963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c8341e94-a3c7-4414-938d-1eaac8b500b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b9bf3229-a5d5-4ad3-a48f-b5b73d1c32d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1b13468d-227f-433d-88a3-4b93e4862340","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T21:18:12.477 INFO:tasks.ceph.ceph_manager.cluster1:[] 2024-04-23T21:18:12.477 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-23T21:18:12.779 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2024-04-23T21:18:12.875 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:18:12.875+0000 7f32c7863640 -1 Falling back to public interface 2024-04-23T21:18:12.886 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:18:12.886+0000 7f9222721640 -1 Falling back to public interface 2024-04-23T21:18:12.895 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:18:12.895+0000 7fc41c210640 -1 Falling back to public interface 2024-04-23T21:18:12.942 INFO:teuthology.misc.health.smithi148.stdout: 2024-04-23T21:18:12.942 INFO:teuthology.misc.health.smithi148.stdout:{"epoch":5,"fsid":"ed086417-ad7c-481e-9d20-547ac50d08e6","created":"2024-04-23T21:18:11.375874+0000","modified":"2024-04-23T21:18:12.261963+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T21:18:12.261963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c8341e94-a3c7-4414-938d-1eaac8b500b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b9bf3229-a5d5-4ad3-a48f-b5b73d1c32d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1b13468d-227f-433d-88a3-4b93e4862340","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T21:18:12.944 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:12.945+0000 7ff052638040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T21:18:12.952 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-04-23T21:18:13.032 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:13.032+0000 7ff052638040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T21:18:13.074 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:13.074+0000 7ff052638040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T21:18:13.238 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:13.239+0000 7ff052638040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T21:18:13.302 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:13.301+0000 7ff052638040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T21:18:13.353 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:13.353+0000 7ff052638040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T21:18:13.466 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:13.467+0000 7ff052638040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T21:18:13.842 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:13.843+0000 7ff052638040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T21:18:14.097 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:14.098+0000 7ff052638040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T21:18:14.148 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:14.149+0000 7ff052638040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T21:18:14.262 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:14.263+0000 7ff052638040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T21:18:14.536 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:14.537+0000 7ff052638040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T21:18:14.585 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:14.585+0000 7ff052638040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T21:18:14.738 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:14.738+0000 7ff052638040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T21:18:14.873 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:14.874+0000 7ff052638040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T21:18:14.922 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:14.923+0000 7ff052638040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T21:18:14.991 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:14.991+0000 7ff052638040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T21:18:15.035 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:15.036+0000 7ff052638040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T21:18:15.511 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:15.511+0000 7ff052638040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T21:18:15.624 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:15.625+0000 7ff052638040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T21:18:16.422 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:18:16.422+0000 7f9222721640 -1 osd.2 0 log_to_monitors true 2024-04-23T21:18:16.453 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:18:16.454+0000 7f32c7863640 -1 osd.1 0 log_to_monitors true 2024-04-23T21:18:16.465 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:18:16.466+0000 7fc41c210640 -1 osd.0 0 log_to_monitors true 2024-04-23T21:18:16.632 INFO:tasks.ceph.cluster1.mgr.x.smithi148.stderr:2024-04-23T21:18:16.633+0000 7feff31d9640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-04-23T21:18:18.678 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:18:18.678+0000 7f921e6b7640 -1 osd.2 0 waiting for initial osdmap 2024-04-23T21:18:18.688 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:18:18.689+0000 7f92164d7640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T21:18:18.712 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:18:18.712+0000 7fc418368640 -1 osd.0 0 waiting for initial osdmap 2024-04-23T21:18:18.724 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:18:18.725+0000 7fc40f974640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T21:18:18.814 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:18:18.815+0000 7f32c37f9640 -1 osd.1 0 waiting for initial osdmap 2024-04-23T21:18:18.830 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:18:18.831+0000 7f32bb619640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T21:18:19.255 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2024-04-23T21:18:19.534 INFO:teuthology.misc.health.smithi148.stdout: 2024-04-23T21:18:19.535 INFO:teuthology.misc.health.smithi148.stdout:{"epoch":7,"fsid":"ed086417-ad7c-481e-9d20-547ac50d08e6","created":"2024-04-23T21:18:11.375874+0000","modified":"2024-04-23T21:18:17.683670+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T21:18:12.261963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c8341e94-a3c7-4414-938d-1eaac8b500b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b9bf3229-a5d5-4ad3-a48f-b5b73d1c32d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1b13468d-227f-433d-88a3-4b93e4862340","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T21:18:19.546 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-04-23T21:18:25.849 DEBUG:teuthology.orchestra.run.smithi148:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2024-04-23T21:18:26.137 INFO:teuthology.misc.health.smithi148.stdout: 2024-04-23T21:18:26.138 INFO:teuthology.misc.health.smithi148.stdout:{"epoch":12,"fsid":"ed086417-ad7c-481e-9d20-547ac50d08e6","created":"2024-04-23T21:18:11.375874+0000","modified":"2024-04-23T21:18:23.899013+0000","last_up_change":"2024-04-23T21:18:19.688295+0000","last_in_change":"2024-04-23T21:18:12.261963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T21:18:21.645859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8341e94-a3c7-4414-938d-1eaac8b500b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2042282222},{"type":"v1","addr":"172.21.15.148:6817","nonce":2042282222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2042282222},{"type":"v1","addr":"172.21.15.148:6819","nonce":2042282222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2042282222},{"type":"v1","addr":"172.21.15.148:6823","nonce":2042282222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2042282222},{"type":"v1","addr":"172.21.15.148:6821","nonce":2042282222}]},"public_addr":"172.21.15.148:6817/2042282222","cluster_addr":"172.21.15.148:6819/2042282222","heartbeat_back_addr":"172.21.15.148:6823/2042282222","heartbeat_front_addr":"172.21.15.148:6821/2042282222","state":["exists","up"]},{"osd":1,"uuid":"b9bf3229-a5d5-4ad3-a48f-b5b73d1c32d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3232011875},{"type":"v1","addr":"172.21.15.148:6801","nonce":3232011875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3232011875},{"type":"v1","addr":"172.21.15.148:6803","nonce":3232011875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3232011875},{"type":"v1","addr":"172.21.15.148:6807","nonce":3232011875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3232011875},{"type":"v1","addr":"172.21.15.148:6805","nonce":3232011875}]},"public_addr":"172.21.15.148:6801/3232011875","cluster_addr":"172.21.15.148:6803/3232011875","heartbeat_back_addr":"172.21.15.148:6807/3232011875","heartbeat_front_addr":"172.21.15.148:6805/3232011875","state":["exists","up"]},{"osd":2,"uuid":"1b13468d-227f-433d-88a3-4b93e4862340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":4099056617},{"type":"v1","addr":"172.21.15.148:6809","nonce":4099056617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":4099056617},{"type":"v1","addr":"172.21.15.148:6811","nonce":4099056617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":4099056617},{"type":"v1","addr":"172.21.15.148:6815","nonce":4099056617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":4099056617},{"type":"v1","addr":"172.21.15.148:6813","nonce":4099056617}]},"public_addr":"172.21.15.148:6809/4099056617","cluster_addr":"172.21.15.148:6811/4099056617","heartbeat_back_addr":"172.21.15.148:6815/4099056617","heartbeat_front_addr":"172.21.15.148:6813/4099056617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:17.448687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:17.468223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:17.467117+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T21:18:26.149 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-04-23T21:18:26.149 INFO:tasks.ceph:Creating RBD pool 2024-04-23T21:18:26.149 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster cluster1 osd pool create rbd 8 2024-04-23T21:18:26.910 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'rbd' created 2024-04-23T21:18:26.924 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster cluster1 osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-23T21:18:27.916 INFO:teuthology.orchestra.run.smithi148.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-23T21:18:27.936 INFO:tasks.ceph:Starting mds daemons in cluster cluster1... 2024-04-23T21:18:27.936 INFO:tasks.ceph:Waiting until cluster1 daemons up and pgs clean... 2024-04-23T21:18:27.936 INFO:tasks.ceph.ceph_manager.cluster1:waiting for mgr available 2024-04-23T21:18:27.936 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 mgr dump --format=json 2024-04-23T21:18:27.937 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-23T21:18:28.271 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:28.282 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":4190748580},{"type":"v1","addr":"172.21.15.148:6825","nonce":4190748580}]},"active_addr":"172.21.15.148:6825/4190748580","active_change":"2024-04-23T21:18:15.627333+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":3198345244}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":3339100441}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":3626360440}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":4007686925}]}]} 2024-04-23T21:18:28.284 INFO:tasks.ceph.ceph_manager.cluster1:mgr available! 2024-04-23T21:18:28.284 INFO:tasks.ceph.ceph_manager.cluster1:waiting for all up 2024-04-23T21:18:28.285 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2024-04-23T21:18:28.571 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:28.571 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":14,"fsid":"ed086417-ad7c-481e-9d20-547ac50d08e6","created":"2024-04-23T21:18:11.375874+0000","modified":"2024-04-23T21:18:27.911350+0000","last_up_change":"2024-04-23T21:18:19.688295+0000","last_in_change":"2024-04-23T21:18:12.261963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T21:18:21.645859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T21:18:26.434320+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8341e94-a3c7-4414-938d-1eaac8b500b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2042282222},{"type":"v1","addr":"172.21.15.148:6817","nonce":2042282222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2042282222},{"type":"v1","addr":"172.21.15.148:6819","nonce":2042282222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2042282222},{"type":"v1","addr":"172.21.15.148:6823","nonce":2042282222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2042282222},{"type":"v1","addr":"172.21.15.148:6821","nonce":2042282222}]},"public_addr":"172.21.15.148:6817/2042282222","cluster_addr":"172.21.15.148:6819/2042282222","heartbeat_back_addr":"172.21.15.148:6823/2042282222","heartbeat_front_addr":"172.21.15.148:6821/2042282222","state":["exists","up"]},{"osd":1,"uuid":"b9bf3229-a5d5-4ad3-a48f-b5b73d1c32d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3232011875},{"type":"v1","addr":"172.21.15.148:6801","nonce":3232011875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3232011875},{"type":"v1","addr":"172.21.15.148:6803","nonce":3232011875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3232011875},{"type":"v1","addr":"172.21.15.148:6807","nonce":3232011875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3232011875},{"type":"v1","addr":"172.21.15.148:6805","nonce":3232011875}]},"public_addr":"172.21.15.148:6801/3232011875","cluster_addr":"172.21.15.148:6803/3232011875","heartbeat_back_addr":"172.21.15.148:6807/3232011875","heartbeat_front_addr":"172.21.15.148:6805/3232011875","state":["exists","up"]},{"osd":2,"uuid":"1b13468d-227f-433d-88a3-4b93e4862340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":4099056617},{"type":"v1","addr":"172.21.15.148:6809","nonce":4099056617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":4099056617},{"type":"v1","addr":"172.21.15.148:6811","nonce":4099056617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":4099056617},{"type":"v1","addr":"172.21.15.148:6815","nonce":4099056617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":4099056617},{"type":"v1","addr":"172.21.15.148:6813","nonce":4099056617}]},"public_addr":"172.21.15.148:6809/4099056617","cluster_addr":"172.21.15.148:6811/4099056617","heartbeat_back_addr":"172.21.15.148:6815/4099056617","heartbeat_front_addr":"172.21.15.148:6813/4099056617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:17.448687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:17.468223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:17.467117+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T21:18:28.584 INFO:tasks.ceph.ceph_manager.cluster1:all up! 2024-04-23T21:18:28.584 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2024-04-23T21:18:28.868 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:28.868 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":14,"fsid":"ed086417-ad7c-481e-9d20-547ac50d08e6","created":"2024-04-23T21:18:11.375874+0000","modified":"2024-04-23T21:18:27.911350+0000","last_up_change":"2024-04-23T21:18:19.688295+0000","last_in_change":"2024-04-23T21:18:12.261963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T21:18:21.645859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T21:18:26.434320+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8341e94-a3c7-4414-938d-1eaac8b500b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2042282222},{"type":"v1","addr":"172.21.15.148:6817","nonce":2042282222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2042282222},{"type":"v1","addr":"172.21.15.148:6819","nonce":2042282222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2042282222},{"type":"v1","addr":"172.21.15.148:6823","nonce":2042282222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2042282222},{"type":"v1","addr":"172.21.15.148:6821","nonce":2042282222}]},"public_addr":"172.21.15.148:6817/2042282222","cluster_addr":"172.21.15.148:6819/2042282222","heartbeat_back_addr":"172.21.15.148:6823/2042282222","heartbeat_front_addr":"172.21.15.148:6821/2042282222","state":["exists","up"]},{"osd":1,"uuid":"b9bf3229-a5d5-4ad3-a48f-b5b73d1c32d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3232011875},{"type":"v1","addr":"172.21.15.148:6801","nonce":3232011875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3232011875},{"type":"v1","addr":"172.21.15.148:6803","nonce":3232011875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3232011875},{"type":"v1","addr":"172.21.15.148:6807","nonce":3232011875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3232011875},{"type":"v1","addr":"172.21.15.148:6805","nonce":3232011875}]},"public_addr":"172.21.15.148:6801/3232011875","cluster_addr":"172.21.15.148:6803/3232011875","heartbeat_back_addr":"172.21.15.148:6807/3232011875","heartbeat_front_addr":"172.21.15.148:6805/3232011875","state":["exists","up"]},{"osd":2,"uuid":"1b13468d-227f-433d-88a3-4b93e4862340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":4099056617},{"type":"v1","addr":"172.21.15.148:6809","nonce":4099056617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":4099056617},{"type":"v1","addr":"172.21.15.148:6811","nonce":4099056617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":4099056617},{"type":"v1","addr":"172.21.15.148:6815","nonce":4099056617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":4099056617},{"type":"v1","addr":"172.21.15.148:6813","nonce":4099056617}]},"public_addr":"172.21.15.148:6809/4099056617","cluster_addr":"172.21.15.148:6811/4099056617","heartbeat_back_addr":"172.21.15.148:6815/4099056617","heartbeat_front_addr":"172.21.15.148:6813/4099056617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:17.448687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:17.468223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:17.467117+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T21:18:28.882 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.0 flush_pg_stats 2024-04-23T21:18:28.882 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.1 flush_pg_stats 2024-04-23T21:18:28.883 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.2 flush_pg_stats 2024-04-23T21:18:29.018 INFO:teuthology.orchestra.run.smithi148.stdout:34359738372 2024-04-23T21:18:29.018 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.0 2024-04-23T21:18:29.028 INFO:teuthology.orchestra.run.smithi148.stdout:34359738372 2024-04-23T21:18:29.028 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.2 2024-04-23T21:18:29.029 INFO:teuthology.orchestra.run.smithi148.stdout:34359738372 2024-04-23T21:18:29.029 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.1 2024-04-23T21:18:29.325 INFO:teuthology.orchestra.run.smithi148.stdout:34359738371 2024-04-23T21:18:29.338 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738372 got 34359738371 for osd.0 2024-04-23T21:18:29.371 INFO:teuthology.orchestra.run.smithi148.stdout:34359738371 2024-04-23T21:18:29.375 INFO:teuthology.orchestra.run.smithi148.stdout:34359738371 2024-04-23T21:18:29.385 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738372 got 34359738371 for osd.2 2024-04-23T21:18:29.388 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738372 got 34359738371 for osd.1 2024-04-23T21:18:30.339 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.0 2024-04-23T21:18:30.386 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.2 2024-04-23T21:18:30.388 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.1 2024-04-23T21:18:30.655 INFO:teuthology.orchestra.run.smithi148.stdout:34359738372 2024-04-23T21:18:30.669 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738372 got 34359738372 for osd.0 2024-04-23T21:18:30.669 DEBUG:teuthology.parallel:result is None 2024-04-23T21:18:30.691 INFO:teuthology.orchestra.run.smithi148.stdout:34359738372 2024-04-23T21:18:30.697 INFO:teuthology.orchestra.run.smithi148.stdout:34359738372 2024-04-23T21:18:30.705 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738372 got 34359738372 for osd.2 2024-04-23T21:18:30.705 DEBUG:teuthology.parallel:result is None 2024-04-23T21:18:30.711 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738372 got 34359738372 for osd.1 2024-04-23T21:18:30.711 DEBUG:teuthology.parallel:result is None 2024-04-23T21:18:30.711 INFO:tasks.ceph.ceph_manager.cluster1:waiting for clean 2024-04-23T21:18:30.711 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-04-23T21:18:31.032 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:31.032 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-23T21:18:31.044 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-23T21:18:29.632722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81540,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283033980,"statfs":{"total":289910292480,"available":289826795520,"internally_reserved":0,"allocated":483328,"data_stored":1282004,"data_compressed":5788,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000664"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T21:18:28.922088+0000","last_change":"2024-04-23T21:18:27.927319+0000","last_active":"2024-04-23T21:18:28.922088+0000","last_peered":"2024-04-23T21:18:28.922088+0000","last_clean":"2024-04-23T21:18:28.922088+0000","last_became_active":"2024-04-23T21:18:27.926230+0000","last_became_peered":"2024-04-23T21:18:27.926230+0000","last_unstale":"2024-04-23T21:18:28.922088+0000","last_undegraded":"2024-04-23T21:18:28.922088+0000","last_fullsized":"2024-04-23T21:18:28.922088+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:16:09.753590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T21:18:28.922384+0000","last_change":"2024-04-23T21:18:27.929313+0000","last_active":"2024-04-23T21:18:28.922384+0000","last_peered":"2024-04-23T21:18:28.922384+0000","last_clean":"2024-04-23T21:18:28.922384+0000","last_became_active":"2024-04-23T21:18:27.928990+0000","last_became_peered":"2024-04-23T21:18:27.928990+0000","last_unstale":"2024-04-23T21:18:28.922384+0000","last_undegraded":"2024-04-23T21:18:28.922384+0000","last_fullsized":"2024-04-23T21:18:28.922384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:13:49.198809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T21:18:28.922293+0000","last_change":"2024-04-23T21:18:27.926566+0000","last_active":"2024-04-23T21:18:28.922293+0000","last_peered":"2024-04-23T21:18:28.922293+0000","last_clean":"2024-04-23T21:18:28.922293+0000","last_became_active":"2024-04-23T21:18:27.926249+0000","last_became_peered":"2024-04-23T21:18:27.926249+0000","last_unstale":"2024-04-23T21:18:28.922293+0000","last_undegraded":"2024-04-23T21:18:28.922293+0000","last_fullsized":"2024-04-23T21:18:28.922293+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:11:22.863696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T21:18:28.922209+0000","last_change":"2024-04-23T21:18:27.926922+0000","last_active":"2024-04-23T21:18:28.922209+0000","last_peered":"2024-04-23T21:18:28.922209+0000","last_clean":"2024-04-23T21:18:28.922209+0000","last_became_active":"2024-04-23T21:18:27.926470+0000","last_became_peered":"2024-04-23T21:18:27.926470+0000","last_unstale":"2024-04-23T21:18:28.922209+0000","last_undegraded":"2024-04-23T21:18:28.922209+0000","last_fullsized":"2024-04-23T21:18:28.922209+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:18:14.714407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:18:27.931758+0000","last_change":"2024-04-23T21:18:27.929109+0000","last_active":"2024-04-23T21:18:27.931758+0000","last_peered":"2024-04-23T21:18:27.931758+0000","last_clean":"2024-04-23T21:18:27.931758+0000","last_became_active":"2024-04-23T21:18:27.928756+0000","last_became_peered":"2024-04-23T21:18:27.928756+0000","last_unstale":"2024-04-23T21:18:27.931758+0000","last_undegraded":"2024-04-23T21:18:27.931758+0000","last_fullsized":"2024-04-23T21:18:27.931758+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:56:25.785120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:18:27.931672+0000","last_change":"2024-04-23T21:18:27.929903+0000","last_active":"2024-04-23T21:18:27.931672+0000","last_peered":"2024-04-23T21:18:27.931672+0000","last_clean":"2024-04-23T21:18:27.931672+0000","last_became_active":"2024-04-23T21:18:27.929576+0000","last_became_peered":"2024-04-23T21:18:27.929576+0000","last_unstale":"2024-04-23T21:18:27.931672+0000","last_undegraded":"2024-04-23T21:18:27.931672+0000","last_fullsized":"2024-04-23T21:18:27.931672+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:33.510017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:18:27.931698+0000","last_change":"2024-04-23T21:18:27.929991+0000","last_active":"2024-04-23T21:18:27.931698+0000","last_peered":"2024-04-23T21:18:27.931698+0000","last_clean":"2024-04-23T21:18:27.931698+0000","last_became_active":"2024-04-23T21:18:27.929679+0000","last_became_peered":"2024-04-23T21:18:27.929679+0000","last_unstale":"2024-04-23T21:18:27.931698+0000","last_undegraded":"2024-04-23T21:18:27.931698+0000","last_fullsized":"2024-04-23T21:18:27.931698+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:58:46.817144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T21:18:28.922162+0000","last_change":"2024-04-23T21:18:27.926735+0000","last_active":"2024-04-23T21:18:28.922162+0000","last_peered":"2024-04-23T21:18:28.922162+0000","last_clean":"2024-04-23T21:18:28.922162+0000","last_became_active":"2024-04-23T21:18:27.926281+0000","last_became_peered":"2024-04-23T21:18:27.926281+0000","last_unstale":"2024-04-23T21:18:28.922162+0000","last_undegraded":"2024-04-23T21:18:28.922162+0000","last_fullsized":"2024-04-23T21:18:28.922162+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:17:26.651368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":145,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T21:18:28.921942+0000","last_change":"2024-04-23T21:18:22.904282+0000","last_active":"2024-04-23T21:18:28.921942+0000","last_peered":"2024-04-23T21:18:28.921942+0000","last_clean":"2024-04-23T21:18:28.921942+0000","last_became_active":"2024-04-23T21:18:22.903255+0000","last_became_peered":"2024-04-23T21:18:22.903255+0000","last_unstale":"2024-04-23T21:18:28.921942+0000","last_undegraded":"2024-04-23T21:18:28.921942+0000","last_fullsized":"2024-04-23T21:18:28.921942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:21.893219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:21.893219+0000","last_clean_scrub_stamp":"2024-04-23T21:18:21.893219+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:17:55.065914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":5788,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27140,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344700,"statfs":{"total":96636764160,"available":96608972800,"internally_reserved":0,"allocated":122880,"data_stored":33756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":624124,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":624124,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T21:18:31.045 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-04-23T21:18:31.323 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:31.323 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-23T21:18:31.335 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-23T21:18:29.632722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81540,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283033980,"statfs":{"total":289910292480,"available":289826795520,"internally_reserved":0,"allocated":483328,"data_stored":1282004,"data_compressed":5788,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000664"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T21:18:28.922088+0000","last_change":"2024-04-23T21:18:27.927319+0000","last_active":"2024-04-23T21:18:28.922088+0000","last_peered":"2024-04-23T21:18:28.922088+0000","last_clean":"2024-04-23T21:18:28.922088+0000","last_became_active":"2024-04-23T21:18:27.926230+0000","last_became_peered":"2024-04-23T21:18:27.926230+0000","last_unstale":"2024-04-23T21:18:28.922088+0000","last_undegraded":"2024-04-23T21:18:28.922088+0000","last_fullsized":"2024-04-23T21:18:28.922088+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:16:09.753590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T21:18:28.922384+0000","last_change":"2024-04-23T21:18:27.929313+0000","last_active":"2024-04-23T21:18:28.922384+0000","last_peered":"2024-04-23T21:18:28.922384+0000","last_clean":"2024-04-23T21:18:28.922384+0000","last_became_active":"2024-04-23T21:18:27.928990+0000","last_became_peered":"2024-04-23T21:18:27.928990+0000","last_unstale":"2024-04-23T21:18:28.922384+0000","last_undegraded":"2024-04-23T21:18:28.922384+0000","last_fullsized":"2024-04-23T21:18:28.922384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:13:49.198809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T21:18:28.922293+0000","last_change":"2024-04-23T21:18:27.926566+0000","last_active":"2024-04-23T21:18:28.922293+0000","last_peered":"2024-04-23T21:18:28.922293+0000","last_clean":"2024-04-23T21:18:28.922293+0000","last_became_active":"2024-04-23T21:18:27.926249+0000","last_became_peered":"2024-04-23T21:18:27.926249+0000","last_unstale":"2024-04-23T21:18:28.922293+0000","last_undegraded":"2024-04-23T21:18:28.922293+0000","last_fullsized":"2024-04-23T21:18:28.922293+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:11:22.863696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T21:18:28.922209+0000","last_change":"2024-04-23T21:18:27.926922+0000","last_active":"2024-04-23T21:18:28.922209+0000","last_peered":"2024-04-23T21:18:28.922209+0000","last_clean":"2024-04-23T21:18:28.922209+0000","last_became_active":"2024-04-23T21:18:27.926470+0000","last_became_peered":"2024-04-23T21:18:27.926470+0000","last_unstale":"2024-04-23T21:18:28.922209+0000","last_undegraded":"2024-04-23T21:18:28.922209+0000","last_fullsized":"2024-04-23T21:18:28.922209+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:18:14.714407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:18:27.931758+0000","last_change":"2024-04-23T21:18:27.929109+0000","last_active":"2024-04-23T21:18:27.931758+0000","last_peered":"2024-04-23T21:18:27.931758+0000","last_clean":"2024-04-23T21:18:27.931758+0000","last_became_active":"2024-04-23T21:18:27.928756+0000","last_became_peered":"2024-04-23T21:18:27.928756+0000","last_unstale":"2024-04-23T21:18:27.931758+0000","last_undegraded":"2024-04-23T21:18:27.931758+0000","last_fullsized":"2024-04-23T21:18:27.931758+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:56:25.785120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:18:27.931672+0000","last_change":"2024-04-23T21:18:27.929903+0000","last_active":"2024-04-23T21:18:27.931672+0000","last_peered":"2024-04-23T21:18:27.931672+0000","last_clean":"2024-04-23T21:18:27.931672+0000","last_became_active":"2024-04-23T21:18:27.929576+0000","last_became_peered":"2024-04-23T21:18:27.929576+0000","last_unstale":"2024-04-23T21:18:27.931672+0000","last_undegraded":"2024-04-23T21:18:27.931672+0000","last_fullsized":"2024-04-23T21:18:27.931672+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:33.510017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:18:27.931698+0000","last_change":"2024-04-23T21:18:27.929991+0000","last_active":"2024-04-23T21:18:27.931698+0000","last_peered":"2024-04-23T21:18:27.931698+0000","last_clean":"2024-04-23T21:18:27.931698+0000","last_became_active":"2024-04-23T21:18:27.929679+0000","last_became_peered":"2024-04-23T21:18:27.929679+0000","last_unstale":"2024-04-23T21:18:27.931698+0000","last_undegraded":"2024-04-23T21:18:27.931698+0000","last_fullsized":"2024-04-23T21:18:27.931698+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:58:46.817144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T21:18:28.922162+0000","last_change":"2024-04-23T21:18:27.926735+0000","last_active":"2024-04-23T21:18:28.922162+0000","last_peered":"2024-04-23T21:18:28.922162+0000","last_clean":"2024-04-23T21:18:28.922162+0000","last_became_active":"2024-04-23T21:18:27.926281+0000","last_became_peered":"2024-04-23T21:18:27.926281+0000","last_unstale":"2024-04-23T21:18:28.922162+0000","last_undegraded":"2024-04-23T21:18:28.922162+0000","last_fullsized":"2024-04-23T21:18:28.922162+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:17:26.651368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":145,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T21:18:28.921942+0000","last_change":"2024-04-23T21:18:22.904282+0000","last_active":"2024-04-23T21:18:28.921942+0000","last_peered":"2024-04-23T21:18:28.921942+0000","last_clean":"2024-04-23T21:18:28.921942+0000","last_became_active":"2024-04-23T21:18:22.903255+0000","last_became_peered":"2024-04-23T21:18:22.903255+0000","last_unstale":"2024-04-23T21:18:28.921942+0000","last_undegraded":"2024-04-23T21:18:28.921942+0000","last_fullsized":"2024-04-23T21:18:28.921942+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:21.893219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:21.893219+0000","last_clean_scrub_stamp":"2024-04-23T21:18:21.893219+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:17:55.065914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":5788,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27140,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344700,"statfs":{"total":96636764160,"available":96608972800,"internally_reserved":0,"allocated":122880,"data_stored":33756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":624124,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":624124,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T21:18:31.336 INFO:tasks.ceph.ceph_manager.cluster1:clean! 2024-04-23T21:18:31.336 INFO:tasks.ceph:Waiting until ceph cluster cluster1 is healthy... 2024-04-23T21:18:31.336 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy 2024-04-23T21:18:31.336 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 health --format=json 2024-04-23T21:18:31.654 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:31.654 INFO:teuthology.orchestra.run.smithi148.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T21:18:31.667 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy done 2024-04-23T21:18:31.667 INFO:teuthology.run_tasks:Running task ceph... 2024-04-23T21:18:31.677 INFO:tasks.ceph:Creating ceph cluster cluster2... 2024-04-23T21:18:31.677 INFO:tasks.ceph:config {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'cluster2', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-23T21:18:31.677 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi/7670210', 'branch': 'main', 'description': 'rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_latest} workloads/rbd-mirror-journal-stress-workunit}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7670210', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'run two ceph clusters and install rbd-mirror'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': '2 ceph clusters with 1 mon and 3 osds each'}, {'desc': 'run four rbd-mirror daemons per cluster'}, {'desc': 'run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon'}], 'name': 'teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'sha1': '8254cb5a247caa8e3d35e534a1fefe2362a0a4b8'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8254cb5a247caa8e3d35e534a1fefe2362a0a4b8'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 950, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'], ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']], 'seed': 4807, 'sha1': '8254cb5a247caa8e3d35e534a1fefe2362a0a4b8', 'sleep_before_teardown': 0, 'subset': '19/128', 'suite': 'rbd', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f', 'targets': {'smithi148.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMX5JWB1fO38Pml0tJfiB3TVJ7QYRvuufJe0BcdYHxgyd5yTnCGYAITfnpWXKHm3rywGVVbfw5bTG0hTY1OpztA=', 'smithi150.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBS9ZCMITkD7l57bj+JwW3X3sW3qR6Mz57CYuRA4xv0lgLTn9nZ1ZGAs6/+IyWptTbmmMLbQ/Re0zEwOe4t0Brk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': ['rbd-mirror'], 'flavor': 'default', 'sha1': '8254cb5a247caa8e3d35e534a1fefe2362a0a4b8'}}, {'ceph': {'cluster': 'cluster1', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'sha1': '8254cb5a247caa8e3d35e534a1fefe2362a0a4b8'}}, {'ceph': {'cluster': 'cluster2', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'sha1': '8254cb5a247caa8e3d35e534a1fefe2362a0a4b8'}}, {'exec': {'cluster1.client.mirror.0': ["sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'"]}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.3', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.3', 'thrash': True}}, {'rbd-mirror-thrash': {'cluster': 'cluster1'}}, {'rbd-mirror-thrash': {'cluster': 'cluster2'}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/rbd_mirror_stress.sh']}, 'env': {'CEPH_ARGS': '', 'MIRROR_POOL_MODE': 'pool', 'RBD_MIRROR_INSTANCES': '4', 'RBD_MIRROR_MODE': 'journal', 'RBD_MIRROR_USE_EXISTING_CLUSTER': '1', 'RBD_MIRROR_USE_RBD_MIRROR': '1'}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-23_20:16:13', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-23T21:18:31.678 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2024-04-23T21:18:31.681 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2024-04-23T21:18:31.698 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m0777 -- /var/run/ceph 2024-04-23T21:18:31.738 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/run/ceph 2024-04-23T21:18:31.768 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:31.769 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T21:18:31.826 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T21:18:31.827 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-04-23T21:18:31.884 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T21:18:31.884 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T21:18:31.884 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-23T21:18:31.884 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T21:18:31.884 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T21:18:31.884 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-23 21:17:50.653138748 +0000 2024-04-23T21:18:31.884 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-23 21:14:57.680988693 +0000 2024-04-23T21:18:31.884 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-23 21:14:57.680988693 +0000 2024-04-23T21:18:31.884 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-23 21:14:57.680988693 +0000 2024-04-23T21:18:31.884 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T21:18:31.949 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-23T21:18:31.949 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-23T21:18:31.949 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.00021354 s, 2.4 MB/s 2024-04-23T21:18:31.950 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T21:18:32.008 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-04-23T21:18:32.066 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T21:18:32.066 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T21:18:32.066 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-23T21:18:32.066 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T21:18:32.066 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T21:18:32.066 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-23 21:17:50.654138729 +0000 2024-04-23T21:18:32.066 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-23 21:14:58.024981534 +0000 2024-04-23T21:18:32.067 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-23 21:14:58.024981534 +0000 2024-04-23T21:18:32.067 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-23 21:14:58.024981534 +0000 2024-04-23T21:18:32.067 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T21:18:32.130 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-23T21:18:32.130 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-23T21:18:32.130 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000217597 s, 2.4 MB/s 2024-04-23T21:18:32.132 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T21:18:32.188 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-04-23T21:18:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T21:18:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T21:18:32.243 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-23T21:18:32.243 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T21:18:32.243 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T21:18:32.243 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-23 21:17:50.654138729 +0000 2024-04-23T21:18:32.243 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-23 21:14:58.379974146 +0000 2024-04-23T21:18:32.243 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-23 21:14:58.379974146 +0000 2024-04-23T21:18:32.243 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-23 21:14:58.379974146 +0000 2024-04-23T21:18:32.244 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T21:18:32.306 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-23T21:18:32.306 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-23T21:18:32.306 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000232693 s, 2.2 MB/s 2024-04-23T21:18:32.308 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T21:18:32.363 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-04-23T21:18:32.418 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T21:18:32.419 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T21:18:32.419 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-23T21:18:32.419 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T21:18:32.419 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T21:18:32.419 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-23 21:17:50.654138729 +0000 2024-04-23T21:18:32.419 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-23 21:14:58.710967257 +0000 2024-04-23T21:18:32.419 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-23 21:14:58.710967257 +0000 2024-04-23T21:18:32.419 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-23 21:14:58.710967257 +0000 2024-04-23T21:18:32.419 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T21:18:32.481 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-23T21:18:32.481 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-23T21:18:32.481 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000191614 s, 2.7 MB/s 2024-04-23T21:18:32.482 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T21:18:32.538 INFO:tasks.ceph:osd dev map: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2024-04-23T21:18:32.538 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'}} 2024-04-23T21:18:32.539 INFO:tasks.ceph:Generating config... 2024-04-23T21:18:32.540 INFO:tasks.ceph:[client] debug rbd = 20 2024-04-23T21:18:32.540 INFO:tasks.ceph:[client] debug rbd_mirror = 15 2024-04-23T21:18:32.540 INFO:tasks.ceph:[client] log to stderr = False 2024-04-23T21:18:32.540 INFO:tasks.ceph:[client] rbd default features = 125 2024-04-23T21:18:32.540 INFO:tasks.ceph:[client] rbd mirror image policy type = simple 2024-04-23T21:18:32.540 INFO:tasks.ceph:[client.mirror.0] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:18:32.540 INFO:tasks.ceph:[client.mirror.0] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:18:32.540 INFO:tasks.ceph:[client.mirror.1] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:18:32.540 INFO:tasks.ceph:[client.mirror.1] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:18:32.540 INFO:tasks.ceph:[client.mirror.2] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:18:32.540 INFO:tasks.ceph:[client.mirror.2] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:18:32.540 INFO:tasks.ceph:[client.mirror.3] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:18:32.540 INFO:tasks.ceph:[client.mirror.3] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:18:32.541 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-04-23T21:18:32.541 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-04-23T21:18:32.541 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-04-23T21:18:32.541 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-23T21:18:32.541 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-23T21:18:32.541 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-23T21:18:32.541 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-23T21:18:32.541 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-23T21:18:32.541 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-23T21:18:32.541 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-04-23T21:18:32.541 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-04-23T21:18:32.541 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-23T21:18:32.541 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-23T21:18:32.541 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-23T21:18:32.542 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-23T21:18:32.542 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-23T21:18:32.542 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-23T21:18:32.542 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-23T21:18:32.542 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-23T21:18:32.542 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-23T21:18:32.542 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-23T21:18:32.543 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-23T21:18:32.543 INFO:tasks.ceph:Setting up cluster2.mon.a... 2024-04-23T21:18:32.543 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster2.keyring 2024-04-23T21:18:32.616 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster2.keyring 2024-04-23T21:18:32.619 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster2.keyring 2024-04-23T21:18:32.700 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2024-04-23T21:18:32.763 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster2.mon.a', '172.21.15.150')] 2024-04-23T21:18:32.764 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.150', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'mon.a': {}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}} 2024-04-23T21:18:32.764 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:32.764 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-23T21:18:32.817 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.150 --print /home/ubuntu/cephtest/cluster2.monmap 2024-04-23T21:18:32.896 INFO:teuthology.orchestra.run.smithi150.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster2.monmap 2024-04-23T21:18:32.896 INFO:teuthology.orchestra.run.smithi150.stdout:monmaptool: generated fsid 04592e9d-e1b9-4919-8817-70bceab06a1f 2024-04-23T21:18:32.897 INFO:teuthology.orchestra.run.smithi150.stdout:setting min_mon_release = quincy 2024-04-23T21:18:32.897 INFO:teuthology.orchestra.run.smithi150.stdout:epoch 0 2024-04-23T21:18:32.897 INFO:teuthology.orchestra.run.smithi150.stdout:fsid 04592e9d-e1b9-4919-8817-70bceab06a1f 2024-04-23T21:18:32.897 INFO:teuthology.orchestra.run.smithi150.stdout:last_changed 2024-04-23T21:18:32.897466+0000 2024-04-23T21:18:32.897 INFO:teuthology.orchestra.run.smithi150.stdout:created 2024-04-23T21:18:32.897466+0000 2024-04-23T21:18:32.897 INFO:teuthology.orchestra.run.smithi150.stdout:min_mon_release 17 (quincy) 2024-04-23T21:18:32.897 INFO:teuthology.orchestra.run.smithi150.stdout:election_strategy: 1 2024-04-23T21:18:32.897 INFO:teuthology.orchestra.run.smithi150.stdout:0: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.a 2024-04-23T21:18:32.897 INFO:teuthology.orchestra.run.smithi150.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster2.monmap (1 monitors) 2024-04-23T21:18:32.898 DEBUG:teuthology.orchestra.run.smithi150:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-23T21:18:32.952 INFO:tasks.ceph:Writing /etc/ceph/cluster2.conf for FSID 04592e9d-e1b9-4919-8817-70bceab06a1f... 2024-04-23T21:18:32.953 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2024-04-23T21:18:32.956 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2024-04-23T21:18:33.002 INFO:teuthology.orchestra.run.smithi148.stdout:[global] 2024-04-23T21:18:33.002 INFO:teuthology.orchestra.run.smithi148.stdout: chdir = "" 2024-04-23T21:18:33.002 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T21:18:33.002 INFO:teuthology.orchestra.run.smithi148.stdout: auth supported = cephx 2024-04-23T21:18:33.002 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.002 INFO:teuthology.orchestra.run.smithi148.stdout: filestore xattr use omap = true 2024-04-23T21:18:33.002 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.002 INFO:teuthology.orchestra.run.smithi148.stdout: mon clock drift allowed = 1.000 2024-04-23T21:18:33.003 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.003 INFO:teuthology.orchestra.run.smithi148.stdout: osd crush chooseleaf type = 0 2024-04-23T21:18:33.003 INFO:teuthology.orchestra.run.smithi148.stdout: auth debug = true 2024-04-23T21:18:33.003 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.003 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on old message = true 2024-04-23T21:18:33.003 INFO:teuthology.orchestra.run.smithi148.stdout: ms die on bug = true 2024-04-23T21:18:33.003 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.003 INFO:teuthology.orchestra.run.smithi148.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T21:18:33.003 INFO:teuthology.orchestra.run.smithi148.stdout: mon pg warn max object skew = 0 2024-04-23T21:18:33.003 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.004 INFO:teuthology.orchestra.run.smithi148.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T21:18:33.004 INFO:teuthology.orchestra.run.smithi148.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T21:18:33.004 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.004 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default size = 2 2024-04-23T21:18:33.004 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.004 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow primary affinity = true 2024-04-23T21:18:33.004 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd allow pg remap = true 2024-04-23T21:18:33.004 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on legacy crush tunables = false 2024-04-23T21:18:33.004 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on crush straw calc version zero = false 2024-04-23T21:18:33.004 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on no sortbitwise = false 2024-04-23T21:18:33.004 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on osd down out interval zero = false 2024-04-23T21:18:33.004 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on too few osds = false 2024-04-23T21:18:33.004 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T21:18:33.004 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T21:18:33.005 INFO:teuthology.orchestra.run.smithi148.stdout: mon_allow_pool_size_one = true 2024-04-23T21:18:33.005 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.005 INFO:teuthology.orchestra.run.smithi148.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T21:18:33.005 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.005 INFO:teuthology.orchestra.run.smithi148.stdout: osd default data pool replay window = 5 2024-04-23T21:18:33.005 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.005 INFO:teuthology.orchestra.run.smithi148.stdout: mon allow pool delete = true 2024-04-23T21:18:33.005 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.005 INFO:teuthology.orchestra.run.smithi148.stdout: mon cluster log file level = debug 2024-04-23T21:18:33.005 INFO:teuthology.orchestra.run.smithi148.stdout: debug asserts on shutdown = true 2024-04-23T21:18:33.005 INFO:teuthology.orchestra.run.smithi148.stdout: mon health detail to clog = false 2024-04-23T21:18:33.005 INFO:teuthology.orchestra.run.smithi148.stdout: mon host = 172.21.15.150 2024-04-23T21:18:33.006 INFO:teuthology.orchestra.run.smithi148.stdout: mon client directed command retry = 5 2024-04-23T21:18:33.006 INFO:teuthology.orchestra.run.smithi148.stdout: mon warn on pool no app = False 2024-04-23T21:18:33.006 INFO:teuthology.orchestra.run.smithi148.stdout: ms inject socket failures = 5000 2024-04-23T21:18:33.006 INFO:teuthology.orchestra.run.smithi148.stdout: fsid = 04592e9d-e1b9-4919-8817-70bceab06a1f 2024-04-23T21:18:33.006 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.006 INFO:teuthology.orchestra.run.smithi148.stdout:[osd] 2024-04-23T21:18:33.006 INFO:teuthology.orchestra.run.smithi148.stdout: osd journal size = 100 2024-04-23T21:18:33.006 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.007 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub load threshold = 5.0 2024-04-23T21:18:33.007 INFO:teuthology.orchestra.run.smithi148.stdout: osd scrub max interval = 600 2024-04-23T21:18:33.007 INFO:teuthology.orchestra.run.smithi148.stdout: osd mclock profile = high_recovery_ops 2024-04-23T21:18:33.007 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.007 INFO:teuthology.orchestra.run.smithi148.stdout: osd recover clone overlap = true 2024-04-23T21:18:33.007 INFO:teuthology.orchestra.run.smithi148.stdout: osd recovery max chunk = 1048576 2024-04-23T21:18:33.007 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.007 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug shutdown = true 2024-04-23T21:18:33.007 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug op order = true 2024-04-23T21:18:33.007 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug verify stray on activate = true 2024-04-23T21:18:33.007 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug trim objects = true 2024-04-23T21:18:33.007 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.007 INFO:teuthology.orchestra.run.smithi148.stdout: osd open classes on start = true 2024-04-23T21:18:33.007 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug pg log writeout = true 2024-04-23T21:18:33.008 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.008 INFO:teuthology.orchestra.run.smithi148.stdout: osd deep scrub update digest min age = 30 2024-04-23T21:18:33.008 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.008 INFO:teuthology.orchestra.run.smithi148.stdout: osd map max advance = 10 2024-04-23T21:18:33.008 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.008 INFO:teuthology.orchestra.run.smithi148.stdout: journal zero on create = true 2024-04-23T21:18:33.008 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.008 INFO:teuthology.orchestra.run.smithi148.stdout: filestore ondisk finisher threads = 3 2024-04-23T21:18:33.008 INFO:teuthology.orchestra.run.smithi148.stdout: filestore apply finisher threads = 3 2024-04-23T21:18:33.008 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.008 INFO:teuthology.orchestra.run.smithi148.stdout: bdev debug aio = true 2024-04-23T21:18:33.009 INFO:teuthology.orchestra.run.smithi148.stdout: osd debug misdirected ops = true 2024-04-23T21:18:33.009 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore block size = 96636764160 2024-04-23T21:18:33.009 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore compression algorithm = zstd 2024-04-23T21:18:33.009 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore compression mode = aggressive 2024-04-23T21:18:33.009 INFO:teuthology.orchestra.run.smithi148.stdout: bluestore fsck on mount = True 2024-04-23T21:18:33.009 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluefs = 1/20 2024-04-23T21:18:33.009 INFO:teuthology.orchestra.run.smithi148.stdout: debug bluestore = 1/20 2024-04-23T21:18:33.009 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-23T21:18:33.009 INFO:teuthology.orchestra.run.smithi148.stdout: debug osd = 20 2024-04-23T21:18:33.009 INFO:teuthology.orchestra.run.smithi148.stdout: debug rocksdb = 4/10 2024-04-23T21:18:33.009 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T21:18:33.009 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd full ratio = 0.9 2024-04-23T21:18:33.010 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd nearfull ratio = 0.8 2024-04-23T21:18:33.010 INFO:teuthology.orchestra.run.smithi148.stdout: osd failsafe full ratio = 0.95 2024-04-23T21:18:33.010 INFO:teuthology.orchestra.run.smithi148.stdout: osd objectstore = bluestore 2024-04-23T21:18:33.010 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.010 INFO:teuthology.orchestra.run.smithi148.stdout:[mgr] 2024-04-23T21:18:33.010 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-23T21:18:33.010 INFO:teuthology.orchestra.run.smithi148.stdout: debug mgr = 20 2024-04-23T21:18:33.010 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-04-23T21:18:33.010 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-04-23T21:18:33.010 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-04-23T21:18:33.010 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-04-23T21:18:33.010 INFO:teuthology.orchestra.run.smithi148.stdout: mgr/telemetry/nag = false 2024-04-23T21:18:33.010 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.010 INFO:teuthology.orchestra.run.smithi148.stdout:[mon] 2024-04-23T21:18:33.011 INFO:teuthology.orchestra.run.smithi148.stdout: debug ms = 1 2024-04-23T21:18:33.011 INFO:teuthology.orchestra.run.smithi148.stdout: debug mon = 20 2024-04-23T21:18:33.011 INFO:teuthology.orchestra.run.smithi148.stdout: debug paxos = 20 2024-04-23T21:18:33.011 INFO:teuthology.orchestra.run.smithi148.stdout: debug auth = 20 2024-04-23T21:18:33.011 INFO:teuthology.orchestra.run.smithi148.stdout: mon data avail warn = 5 2024-04-23T21:18:33.011 INFO:teuthology.orchestra.run.smithi148.stdout: mon mgr mkfs grace = 240 2024-04-23T21:18:33.011 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min pgs per osd = 4 2024-04-23T21:18:33.011 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd reporter subtree level = osd 2024-04-23T21:18:33.011 INFO:teuthology.orchestra.run.smithi148.stdout: mon osd prime pg temp = true 2024-04-23T21:18:33.011 INFO:teuthology.orchestra.run.smithi148.stdout: mon reweight min bytes per osd = 10 2024-04-23T21:18:33.011 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.011 INFO:teuthology.orchestra.run.smithi148.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T21:18:33.011 INFO:teuthology.orchestra.run.smithi148.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T21:18:33.011 INFO:teuthology.orchestra.run.smithi148.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T21:18:33.011 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.011 INFO:teuthology.orchestra.run.smithi148.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T21:18:33.012 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T21:18:33.012 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T21:18:33.012 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.012 INFO:teuthology.orchestra.run.smithi148.stdout: # 1m isn't quite enough 2024-04-23T21:18:33.012 INFO:teuthology.orchestra.run.smithi148.stdout: mon_down_mkfs_grace = 2m 2024-04-23T21:18:33.012 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.012 INFO:teuthology.orchestra.run.smithi148.stdout: mon_warn_on_filestore_osds = false 2024-04-23T21:18:33.012 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:18:33.012 INFO:teuthology.orchestra.run.smithi148.stdout:[client] 2024-04-23T21:18:33.012 INFO:teuthology.orchestra.run.smithi148.stdout: rgw cache enabled = true 2024-04-23T21:18:33.012 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable ops log = true 2024-04-23T21:18:33.012 INFO:teuthology.orchestra.run.smithi148.stdout: rgw enable usage log = true 2024-04-23T21:18:33.012 INFO:teuthology.orchestra.run.smithi148.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T21:18:33.013 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T21:18:33.013 INFO:teuthology.orchestra.run.smithi148.stdout: debug rbd = 20 2024-04-23T21:18:33.013 INFO:teuthology.orchestra.run.smithi148.stdout: debug rbd_mirror = 15 2024-04-23T21:18:33.013 INFO:teuthology.orchestra.run.smithi148.stdout: log to stderr = False 2024-04-23T21:18:33.013 INFO:teuthology.orchestra.run.smithi148.stdout: rbd default features = 125 2024-04-23T21:18:33.013 INFO:teuthology.orchestra.run.smithi148.stdout: rbd mirror image policy type = simple 2024-04-23T21:18:33.013 INFO:teuthology.orchestra.run.smithi148.stdout:[mon.a] 2024-04-23T21:18:33.013 INFO:teuthology.orchestra.run.smithi148.stdout:[client.mirror.0] 2024-04-23T21:18:33.013 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:18:33.013 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:18:33.013 INFO:teuthology.orchestra.run.smithi148.stdout:[client.mirror.1] 2024-04-23T21:18:33.014 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:18:33.014 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:18:33.014 INFO:teuthology.orchestra.run.smithi148.stdout:[client.mirror.2] 2024-04-23T21:18:33.014 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:18:33.014 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:18:33.014 INFO:teuthology.orchestra.run.smithi148.stdout:[client.mirror.3] 2024-04-23T21:18:33.014 INFO:teuthology.orchestra.run.smithi148.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:18:33.014 INFO:teuthology.orchestra.run.smithi148.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:18:33.033 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2024-04-23T21:18:33.033 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2024-04-23T21:18:33.033 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-23T21:18:33.033 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2024-04-23T21:18:33.033 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.033 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2024-04-23T21:18:33.033 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.033 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2024-04-23T21:18:33.033 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.033 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2024-04-23T21:18:33.033 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2024-04-23T21:18:33.033 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.034 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2024-04-23T21:18:33.034 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2024-04-23T21:18:33.034 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.034 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-23T21:18:33.034 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2024-04-23T21:18:33.034 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.034 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2024-04-23T21:18:33.034 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-23T21:18:33.034 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.034 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2024-04-23T21:18:33.034 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.034 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2024-04-23T21:18:33.034 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2024-04-23T21:18:33.034 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2024-04-23T21:18:33.035 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2024-04-23T21:18:33.035 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2024-04-23T21:18:33.035 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2024-04-23T21:18:33.035 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2024-04-23T21:18:33.035 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-23T21:18:33.035 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-23T21:18:33.035 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2024-04-23T21:18:33.035 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.035 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-23T21:18:33.035 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.035 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2024-04-23T21:18:33.035 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.035 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2024-04-23T21:18:33.035 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.036 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2024-04-23T21:18:33.036 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2024-04-23T21:18:33.036 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2024-04-23T21:18:33.036 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = 172.21.15.150 2024-04-23T21:18:33.036 INFO:teuthology.orchestra.run.smithi150.stdout: mon client directed command retry = 5 2024-04-23T21:18:33.036 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on pool no app = False 2024-04-23T21:18:33.036 INFO:teuthology.orchestra.run.smithi150.stdout: ms inject socket failures = 5000 2024-04-23T21:18:33.036 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = 04592e9d-e1b9-4919-8817-70bceab06a1f 2024-04-23T21:18:33.036 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.036 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2024-04-23T21:18:33.036 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2024-04-23T21:18:33.036 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.037 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2024-04-23T21:18:33.037 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2024-04-23T21:18:33.037 INFO:teuthology.orchestra.run.smithi150.stdout: osd mclock profile = high_recovery_ops 2024-04-23T21:18:33.037 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.037 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2024-04-23T21:18:33.037 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2024-04-23T21:18:33.037 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.037 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2024-04-23T21:18:33.037 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2024-04-23T21:18:33.037 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2024-04-23T21:18:33.038 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug trim objects = true 2024-04-23T21:18:33.038 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.038 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2024-04-23T21:18:33.038 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2024-04-23T21:18:33.038 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.038 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2024-04-23T21:18:33.038 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.038 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2024-04-23T21:18:33.038 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.038 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2024-04-23T21:18:33.038 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.038 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2024-04-23T21:18:33.039 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2024-04-23T21:18:33.039 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.039 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2024-04-23T21:18:33.039 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2024-04-23T21:18:33.039 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2024-04-23T21:18:33.039 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore compression algorithm = zstd 2024-04-23T21:18:33.039 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore compression mode = aggressive 2024-04-23T21:18:33.039 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2024-04-23T21:18:33.039 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 1/20 2024-04-23T21:18:33.040 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 1/20 2024-04-23T21:18:33.040 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-23T21:18:33.040 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2024-04-23T21:18:33.040 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 4/10 2024-04-23T21:18:33.040 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2024-04-23T21:18:33.040 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2024-04-23T21:18:33.040 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2024-04-23T21:18:33.040 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2024-04-23T21:18:33.040 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2024-04-23T21:18:33.040 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.040 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2024-04-23T21:18:33.040 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-23T21:18:33.040 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2024-04-23T21:18:33.040 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-04-23T21:18:33.041 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-04-23T21:18:33.041 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-04-23T21:18:33.041 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-04-23T21:18:33.041 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2024-04-23T21:18:33.041 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.041 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2024-04-23T21:18:33.041 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2024-04-23T21:18:33.041 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2024-04-23T21:18:33.041 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2024-04-23T21:18:33.041 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2024-04-23T21:18:33.041 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2024-04-23T21:18:33.041 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2024-04-23T21:18:33.041 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2024-04-23T21:18:33.041 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2024-04-23T21:18:33.041 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2024-04-23T21:18:33.042 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2024-04-23T21:18:33.042 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.042 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-23T21:18:33.042 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2024-04-23T21:18:33.042 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2024-04-23T21:18:33.042 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.042 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2024-04-23T21:18:33.042 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-23T21:18:33.042 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-23T21:18:33.042 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.042 INFO:teuthology.orchestra.run.smithi150.stdout: # 1m isn't quite enough 2024-04-23T21:18:33.042 INFO:teuthology.orchestra.run.smithi150.stdout: mon_down_mkfs_grace = 2m 2024-04-23T21:18:33.042 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.042 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_filestore_osds = false 2024-04-23T21:18:33.043 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:33.043 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2024-04-23T21:18:33.043 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2024-04-23T21:18:33.043 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2024-04-23T21:18:33.043 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2024-04-23T21:18:33.043 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-23T21:18:33.043 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-23T21:18:33.043 INFO:teuthology.orchestra.run.smithi150.stdout: debug rbd = 20 2024-04-23T21:18:33.043 INFO:teuthology.orchestra.run.smithi150.stdout: debug rbd_mirror = 15 2024-04-23T21:18:33.043 INFO:teuthology.orchestra.run.smithi150.stdout: log to stderr = False 2024-04-23T21:18:33.043 INFO:teuthology.orchestra.run.smithi150.stdout: rbd default features = 125 2024-04-23T21:18:33.044 INFO:teuthology.orchestra.run.smithi150.stdout: rbd mirror image policy type = simple 2024-04-23T21:18:33.044 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2024-04-23T21:18:33.044 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.0] 2024-04-23T21:18:33.044 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:18:33.044 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:18:33.044 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.1] 2024-04-23T21:18:33.044 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:18:33.044 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:18:33.044 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.2] 2024-04-23T21:18:33.044 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:18:33.044 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:18:33.044 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.3] 2024-04-23T21:18:33.044 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2024-04-23T21:18:33.044 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2024-04-23T21:18:33.045 INFO:tasks.ceph:Creating admin key on cluster2.mon.a... 2024-04-23T21:18:33.045 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster2.keyring 2024-04-23T21:18:33.137 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-23T21:18:33.137 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:33.137 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster2.keyring of=/dev/stdout 2024-04-23T21:18:33.152 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:33.153 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/home/ubuntu/cephtest/cluster2.monmap of=/dev/stdout 2024-04-23T21:18:33.208 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi148.front.sepia.ceph.com 2024-04-23T21:18:33.208 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:18:33.208 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/cluster2.keyring 2024-04-23T21:18:33.208 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2024-04-23T21:18:33.247 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:18:33.247 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/home/ubuntu/cephtest/cluster2.monmap 2024-04-23T21:18:33.305 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi150.front.sepia.ceph.com 2024-04-23T21:18:33.305 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:33.305 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/cluster2.keyring 2024-04-23T21:18:33.305 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2024-04-23T21:18:33.339 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:33.339 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/cluster2.monmap 2024-04-23T21:18:33.394 INFO:tasks.ceph:Setting up mon nodes... 2024-04-23T21:18:33.394 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-23T21:18:33.395 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /var/lib/ceph/mgr/cluster2-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster2-x/keyring 2024-04-23T21:18:33.448 INFO:teuthology.orchestra.run.smithi148.stdout:creating /var/lib/ceph/mgr/cluster2-x/keyring 2024-04-23T21:18:33.451 INFO:tasks.ceph:Setting up mds nodes... 2024-04-23T21:18:33.451 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-23T21:18:33.451 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/cluster2.client.0.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.0.keyring 2024-04-23T21:18:33.493 INFO:teuthology.orchestra.run.smithi148.stdout:creating /etc/ceph/cluster2.client.0.keyring 2024-04-23T21:18:33.507 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster2.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.keyring 2024-04-23T21:18:33.545 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster2.client.mirror.keyring 2024-04-23T21:18:33.557 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster2.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.0.keyring 2024-04-23T21:18:33.636 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster2.client.mirror.0.keyring 2024-04-23T21:18:33.647 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.1 /etc/ceph/cluster2.client.mirror.1.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.1.keyring 2024-04-23T21:18:33.727 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster2.client.mirror.1.keyring 2024-04-23T21:18:33.738 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.2 /etc/ceph/cluster2.client.mirror.2.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.2.keyring 2024-04-23T21:18:33.817 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster2.client.mirror.2.keyring 2024-04-23T21:18:33.828 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.3 /etc/ceph/cluster2.client.mirror.3.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.3.keyring 2024-04-23T21:18:33.908 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster2.client.mirror.3.keyring 2024-04-23T21:18:33.919 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.4 /etc/ceph/cluster2.client.mirror.4.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.4.keyring 2024-04-23T21:18:34.001 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster2.client.mirror.4.keyring 2024-04-23T21:18:34.012 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.5 /etc/ceph/cluster2.client.mirror.5.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.5.keyring 2024-04-23T21:18:34.091 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster2.client.mirror.5.keyring 2024-04-23T21:18:34.106 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.6 /etc/ceph/cluster2.client.mirror.6.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.6.keyring 2024-04-23T21:18:34.185 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/cluster2.client.mirror.6.keyring 2024-04-23T21:18:34.196 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-23T21:18:34.196 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi148.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'}} 2024-04-23T21:18:34.197 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/cluster2-0 2024-04-23T21:18:34.260 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2024-04-23T21:18:34.260 INFO:tasks.ceph:role: cluster2.osd.0 2024-04-23T21:18:34.260 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-23T21:18:34.260 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-23T21:18:34.330 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T21:18:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T21:18:34.330 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T21:18:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T21:18:34.331 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T21:18:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-04-23T21:18:34.331 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T21:18:34.331 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T21:18:34.331 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T21:18:34.331 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T21:18:34.831 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-04-23T21:18:34.832 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-04-23T21:18:34.833 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/cluster2-0 2024-04-23T21:18:34.959 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-0 2024-04-23T21:18:34.991 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/cluster2-1 2024-04-23T21:18:35.054 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2024-04-23T21:18:35.054 INFO:tasks.ceph:role: cluster2.osd.1 2024-04-23T21:18:35.054 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-23T21:18:35.055 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-23T21:18:35.126 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T21:18:35.126 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T21:18:35.126 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T21:18:35.126 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T21:18:35.126 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T21:18:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-04-23T21:18:35.127 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T21:18:35.127 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T21:18:35.127 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T21:18:35.127 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T21:18:35.588 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-04-23T21:18:35.590 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-04-23T21:18:35.590 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster2-1 2024-04-23T21:18:35.681 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-1 2024-04-23T21:18:35.708 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/cluster2-2 2024-04-23T21:18:35.773 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2024-04-23T21:18:35.773 INFO:tasks.ceph:role: cluster2.osd.2 2024-04-23T21:18:35.773 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-23T21:18:35.773 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-23T21:18:35.843 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-23T21:18:35.844 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-23T21:18:35.844 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-23T21:18:35.844 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-23T21:18:35.844 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-23T21:18:35.844 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2024-04-23T21:18:35.844 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-23T21:18:35.844 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-23T21:18:35.844 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-23T21:18:35.844 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-23T21:18:36.334 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2024-04-23T21:18:36.335 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2024-04-23T21:18:36.336 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster2-2 2024-04-23T21:18:36.460 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-2 2024-04-23T21:18:36.489 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster2.monmap 2024-04-23T21:18:36.589 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:36.590+0000 7f6d1aab5640 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-0/keyring: can't open /var/lib/ceph/osd/cluster2-0/keyring: (2) No such file or directory 2024-04-23T21:18:36.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:36.591+0000 7f6d1aab5640 -1 created new key in keyring /var/lib/ceph/osd/cluster2-0/keyring 2024-04-23T21:18:36.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:36.591+0000 7f6d1aab5640 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2024-04-23T21:18:36.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:36.591+0000 7f6d1aab5640 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2024-04-23T21:18:36.590 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:36.591+0000 7f6d1aab5640 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2024-04-23T21:18:36.597 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:36.598+0000 7f6d1aab5640 -1 bluestore(/var/lib/ceph/osd/cluster2-0) _read_fsid unparsable uuid 2024-04-23T21:18:39.444 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-0 2024-04-23T21:18:39.473 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster2.monmap 2024-04-23T21:18:39.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:39.563+0000 7f835c663640 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-1/keyring: can't open /var/lib/ceph/osd/cluster2-1/keyring: (2) No such file or directory 2024-04-23T21:18:39.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:39.563+0000 7f835c663640 -1 created new key in keyring /var/lib/ceph/osd/cluster2-1/keyring 2024-04-23T21:18:39.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:39.563+0000 7f835c663640 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2024-04-23T21:18:39.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:39.563+0000 7f835c663640 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2024-04-23T21:18:39.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:39.563+0000 7f835c663640 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2024-04-23T21:18:39.563 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:39.564+0000 7f835c663640 -1 bluestore(/var/lib/ceph/osd/cluster2-1) _read_fsid unparsable uuid 2024-04-23T21:18:42.411 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-1 2024-04-23T21:18:42.439 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster2.monmap 2024-04-23T21:18:42.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:42.529+0000 7f2eb321b640 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-2/keyring: can't open /var/lib/ceph/osd/cluster2-2/keyring: (2) No such file or directory 2024-04-23T21:18:42.528 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:42.529+0000 7f2eb321b640 -1 created new key in keyring /var/lib/ceph/osd/cluster2-2/keyring 2024-04-23T21:18:42.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:42.529+0000 7f2eb321b640 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2024-04-23T21:18:42.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:42.529+0000 7f2eb321b640 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2024-04-23T21:18:42.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:42.529+0000 7f2eb321b640 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2024-04-23T21:18:42.529 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-23T21:18:42.530+0000 7f2eb321b640 -1 bluestore(/var/lib/ceph/osd/cluster2-2) _read_fsid unparsable uuid 2024-04-23T21:18:45.379 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-2 2024-04-23T21:18:45.404 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-23T21:18:45.405 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:18:45.405 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/var/lib/ceph/mgr/cluster2-x/keyring of=/dev/stdout 2024-04-23T21:18:45.433 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:45.433 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/cluster2-0/keyring of=/dev/stdout 2024-04-23T21:18:45.470 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:45.470 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/cluster2-1/keyring of=/dev/stdout 2024-04-23T21:18:45.536 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:45.536 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/cluster2-2/keyring of=/dev/stdout 2024-04-23T21:18:45.601 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-04-23T21:18:45.601 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/etc/ceph/cluster2.client.0.keyring of=/dev/stdout 2024-04-23T21:18:45.619 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:45.619 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster2.client.mirror.keyring of=/dev/stdout 2024-04-23T21:18:45.656 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:45.656 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster2.client.mirror.0.keyring of=/dev/stdout 2024-04-23T21:18:45.712 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:45.712 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster2.client.mirror.1.keyring of=/dev/stdout 2024-04-23T21:18:45.767 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:45.767 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster2.client.mirror.2.keyring of=/dev/stdout 2024-04-23T21:18:45.822 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:45.822 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster2.client.mirror.3.keyring of=/dev/stdout 2024-04-23T21:18:45.877 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:45.877 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster2.client.mirror.4.keyring of=/dev/stdout 2024-04-23T21:18:45.932 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:45.932 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster2.client.mirror.5.keyring of=/dev/stdout 2024-04-23T21:18:45.988 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:45.988 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/cluster2.client.mirror.6.keyring of=/dev/stdout 2024-04-23T21:18:46.043 INFO:tasks.ceph:Adding keys to all mons... 2024-04-23T21:18:46.043 DEBUG:teuthology.orchestra.run.smithi150:> sudo tee -a /etc/ceph/cluster2.keyring 2024-04-23T21:18:46.105 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.x] 2024-04-23T21:18:46.106 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCpJShma/fFGhAAwzXbcmJ1MgFWAf7Lgbq7xQ== 2024-04-23T21:18:46.106 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.0] 2024-04-23T21:18:46.106 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCsJShm1Hc9IxAA0MLyAIMJblRjzk/WcK/fJQ== 2024-04-23T21:18:46.106 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.1] 2024-04-23T21:18:46.106 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCvJShmelqdIRAA6VmW5wbXFEenI6U7KE0pxA== 2024-04-23T21:18:46.106 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.2] 2024-04-23T21:18:46.106 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCyJShmZyqTHxAA7FWdVHNqi+yaBKBHKkRd0Q== 2024-04-23T21:18:46.106 INFO:teuthology.orchestra.run.smithi150.stdout:[client.0] 2024-04-23T21:18:46.106 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCpJShmD/50HRAAgiPUKGjmpqRM+x1v3It0wA== 2024-04-23T21:18:46.106 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror] 2024-04-23T21:18:46.106 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCpJShmsJ2dIBAA2A53DeXSvfNJKprwNBgHaw== 2024-04-23T21:18:46.106 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.0] 2024-04-23T21:18:46.106 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCpJShmnxkBJhAAmue1/B93gFR+6eVYamaVHg== 2024-04-23T21:18:46.107 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.1] 2024-04-23T21:18:46.107 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCpJShm08xrKxAA6QtxWMEHkqJFJ6lp+Ho/KA== 2024-04-23T21:18:46.107 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.2] 2024-04-23T21:18:46.107 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCpJShmytHLMBAAW1Urc6BwirEM68rUs2Ozkg== 2024-04-23T21:18:46.107 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.3] 2024-04-23T21:18:46.107 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCpJShmNnQ3NhAAvI3BiwXT15yzYV2kixHEuQ== 2024-04-23T21:18:46.107 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.4] 2024-04-23T21:18:46.107 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCqJShm6JglABAABOpgdxYYr3hMq4N9FPOprA== 2024-04-23T21:18:46.107 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.5] 2024-04-23T21:18:46.107 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCqJShmYfWRBRAAnrCPbNZVAM4GFjVQ/pI3SA== 2024-04-23T21:18:46.107 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.6] 2024-04-23T21:18:46.107 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCqJShmCDAeCxAA7GBUP1UfOSHPcTq/SsyeTw== 2024-04-23T21:18:46.108 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-23T21:18:46.197 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T21:18:46.246 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T21:18:46.294 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-23T21:18:46.343 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:46.394 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:46.445 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:46.496 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:46.547 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:46.598 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:46.649 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.4 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:46.701 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.5 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:46.751 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.6 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-23T21:18:46.802 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-23T21:18:46.803 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mon/cluster2-a 2024-04-23T21:18:46.826 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster2 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster2.monmap --keyring /etc/ceph/cluster2.keyring 2024-04-23T21:18:46.922 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster2-a 2024-04-23T21:18:46.945 DEBUG:teuthology.orchestra.run.smithi150:> rm -- /home/ubuntu/cephtest/cluster2.monmap 2024-04-23T21:18:47.000 INFO:tasks.ceph:Starting mon daemons in cluster cluster2... 2024-04-23T21:18:47.000 INFO:tasks.ceph.cluster2.mon.a:Restarting daemon 2024-04-23T21:18:47.000 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster2 -i a 2024-04-23T21:18:47.042 INFO:tasks.ceph.cluster2.mon.a:Started 2024-04-23T21:18:47.043 INFO:tasks.ceph:Starting mgr daemons in cluster cluster2... 2024-04-23T21:18:47.043 INFO:tasks.ceph.cluster2.mgr.x:Restarting daemon 2024-04-23T21:18:47.043 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster2 -i x 2024-04-23T21:18:47.045 INFO:tasks.ceph.cluster2.mgr.x:Started 2024-04-23T21:18:47.046 INFO:tasks.ceph:Setting crush tunables to default 2024-04-23T21:18:47.046 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster cluster2 osd crush tunables default 2024-04-23T21:18:47.209 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:47.209+0000 7fa8c7646040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T21:18:47.215 INFO:teuthology.orchestra.run.smithi150.stderr:adjusted tunables profile to default 2024-04-23T21:18:47.226 INFO:tasks.ceph:check_enable_crimson: False 2024-04-23T21:18:47.226 INFO:tasks.ceph:Starting osd daemons in cluster cluster2... 2024-04-23T21:18:47.226 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:47.226 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/cluster2-0/fsid of=/dev/stdout 2024-04-23T21:18:47.249 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:47.249 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/cluster2-1/fsid of=/dev/stdout 2024-04-23T21:18:47.312 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:18:47.312 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/cluster2-2/fsid of=/dev/stdout 2024-04-23T21:18:47.321 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:47.321+0000 7fa8c7646040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T21:18:47.368 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:47.368+0000 7fa8c7646040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T21:18:47.375 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster cluster2 osd new e2321b5c-7aac-4fe7-9ec6-30b635f38c83 0 2024-04-23T21:18:47.570 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:47.570+0000 7fa8c7646040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T21:18:47.589 INFO:teuthology.orchestra.run.smithi150.stdout:0 2024-04-23T21:18:47.600 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster cluster2 osd new fad0d87f-9151-4aec-859f-1d499f45e002 1 2024-04-23T21:18:47.613 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:47.614+0000 7fa8c7646040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T21:18:47.774 INFO:teuthology.orchestra.run.smithi150.stdout:1 2024-04-23T21:18:47.785 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster cluster2 osd new b14908e0-1927-45f6-868a-32938df0ef0b 2 2024-04-23T21:18:47.957 INFO:teuthology.orchestra.run.smithi150.stdout:2 2024-04-23T21:18:47.969 INFO:tasks.ceph.cluster2.osd.0:Restarting daemon 2024-04-23T21:18:47.970 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 0 2024-04-23T21:18:47.972 INFO:tasks.ceph.cluster2.osd.0:Started 2024-04-23T21:18:47.972 INFO:tasks.ceph.cluster2.osd.1:Restarting daemon 2024-04-23T21:18:47.972 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 1 2024-04-23T21:18:47.974 INFO:tasks.ceph.cluster2.osd.1:Started 2024-04-23T21:18:47.974 INFO:tasks.ceph.cluster2.osd.2:Restarting daemon 2024-04-23T21:18:47.974 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 2 2024-04-23T21:18:47.976 INFO:tasks.ceph.cluster2.osd.2:Started 2024-04-23T21:18:47.976 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2024-04-23T21:18:48.095 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:48.095+0000 7fa8c7646040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T21:18:48.147 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:18:48.147 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":5,"fsid":"04592e9d-e1b9-4919-8817-70bceab06a1f","created":"2024-04-23T21:18:47.128102+0000","modified":"2024-04-23T21:18:47.958645+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T21:18:47.958645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e2321b5c-7aac-4fe7-9ec6-30b635f38c83","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fad0d87f-9151-4aec-859f-1d499f45e002","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b14908e0-1927-45f6-868a-32938df0ef0b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T21:18:48.158 INFO:tasks.ceph.ceph_manager.cluster2:[] 2024-04-23T21:18:48.158 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-23T21:18:48.178 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:48.179+0000 7fa8c7646040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T21:18:48.221 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:48.221+0000 7fa8c7646040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T21:18:48.383 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:48.383+0000 7fa8c7646040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T21:18:48.428 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:48.428+0000 7fa8c7646040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T21:18:48.460 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2024-04-23T21:18:48.477 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:48.477+0000 7fa8c7646040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T21:18:48.564 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:18:48.565+0000 7f840cc26640 -1 Falling back to public interface 2024-04-23T21:18:48.565 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:18:48.565+0000 7f5884933640 -1 Falling back to public interface 2024-04-23T21:18:48.565 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:18:48.565+0000 7f58838e2640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-04-23T21:18:48.565 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:18:48.565+0000 7f58838e2640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2024-04-23T21:18:48.572 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:18:48.573+0000 7fe98cc11640 -1 Falling back to public interface 2024-04-23T21:18:48.587 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:48.587+0000 7fa8c7646040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T21:18:48.619 INFO:teuthology.misc.health.smithi150.stdout: 2024-04-23T21:18:48.619 INFO:teuthology.misc.health.smithi150.stdout:{"epoch":5,"fsid":"04592e9d-e1b9-4919-8817-70bceab06a1f","created":"2024-04-23T21:18:47.128102+0000","modified":"2024-04-23T21:18:47.958645+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T21:18:47.958645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e2321b5c-7aac-4fe7-9ec6-30b635f38c83","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fad0d87f-9151-4aec-859f-1d499f45e002","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b14908e0-1927-45f6-868a-32938df0ef0b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T21:18:48.629 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-04-23T21:18:48.906 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:48.906+0000 7fa8c7646040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T21:18:49.119 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:49.119+0000 7fa8c7646040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T21:18:49.168 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:49.168+0000 7fa8c7646040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T21:18:49.282 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:49.282+0000 7fa8c7646040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T21:18:49.548 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:49.548+0000 7fa8c7646040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T21:18:49.591 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:49.591+0000 7fa8c7646040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T21:18:49.734 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:49.734+0000 7fa8c7646040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T21:18:49.865 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:49.865+0000 7fa8c7646040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T21:18:49.913 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:49.913+0000 7fa8c7646040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T21:18:49.966 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:49.966+0000 7fa8c7646040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T21:18:50.009 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:50.009+0000 7fa8c7646040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T21:18:50.394 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:50.394+0000 7fa8c7646040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T21:18:50.509 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:50.509+0000 7fa8c7646040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T21:18:51.577 INFO:tasks.ceph.cluster2.mgr.x.smithi148.stderr:2024-04-23T21:18:51.577+0000 7fa868210640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-04-23T21:18:51.978 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:18:51.978+0000 7f840cc26640 -1 osd.0 0 log_to_monitors true 2024-04-23T21:18:52.027 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:18:52.027+0000 7fe98cc11640 -1 osd.2 0 log_to_monitors true 2024-04-23T21:18:52.063 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:18:52.063+0000 7f5884933640 -1 osd.1 0 log_to_monitors true 2024-04-23T21:18:54.572 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:18:54.572+0000 7f8408615640 -1 osd.0 0 waiting for initial osdmap 2024-04-23T21:18:54.585 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:18:54.585+0000 7f8400435640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T21:18:54.593 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:18:54.593+0000 7fe988622640 -1 osd.2 0 waiting for initial osdmap 2024-04-23T21:18:54.602 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:18:54.602+0000 7f58810dd640 -1 osd.1 0 waiting for initial osdmap 2024-04-23T21:18:54.611 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:18:54.611+0000 7fe980442640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T21:18:54.615 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:18:54.615+0000 7f58786e9640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T21:18:54.932 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2024-04-23T21:18:55.206 INFO:teuthology.misc.health.smithi150.stdout: 2024-04-23T21:18:55.206 INFO:teuthology.misc.health.smithi150.stdout:{"epoch":7,"fsid":"04592e9d-e1b9-4919-8817-70bceab06a1f","created":"2024-04-23T21:18:47.128102+0000","modified":"2024-04-23T21:18:53.575536+0000","last_up_change":"0.000000","last_in_change":"2024-04-23T21:18:47.958645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e2321b5c-7aac-4fe7-9ec6-30b635f38c83","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fad0d87f-9151-4aec-859f-1d499f45e002","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b14908e0-1927-45f6-868a-32938df0ef0b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T21:18:55.218 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-04-23T21:19:01.522 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2024-04-23T21:19:01.814 INFO:teuthology.misc.health.smithi150.stdout: 2024-04-23T21:19:01.814 INFO:teuthology.misc.health.smithi150.stdout:{"epoch":12,"fsid":"04592e9d-e1b9-4919-8817-70bceab06a1f","created":"2024-04-23T21:18:47.128102+0000","modified":"2024-04-23T21:18:59.595533+0000","last_up_change":"2024-04-23T21:18:55.580476+0000","last_in_change":"2024-04-23T21:18:47.958645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T21:18:56.733503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2321b5c-7aac-4fe7-9ec6-30b635f38c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4253773404},{"type":"v1","addr":"172.21.15.150:6801","nonce":4253773404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6803","nonce":4253773404},{"type":"v1","addr":"172.21.15.150:6805","nonce":4253773404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6811","nonce":4253773404},{"type":"v1","addr":"172.21.15.150:6813","nonce":4253773404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6807","nonce":4253773404},{"type":"v1","addr":"172.21.15.150:6809","nonce":4253773404}]},"public_addr":"172.21.15.150:6801/4253773404","cluster_addr":"172.21.15.150:6805/4253773404","heartbeat_back_addr":"172.21.15.150:6813/4253773404","heartbeat_front_addr":"172.21.15.150:6809/4253773404","state":["exists","up"]},{"osd":1,"uuid":"fad0d87f-9151-4aec-859f-1d499f45e002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1557391161},{"type":"v1","addr":"172.21.15.150:6804","nonce":1557391161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1557391161},{"type":"v1","addr":"172.21.15.150:6808","nonce":1557391161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1557391161},{"type":"v1","addr":"172.21.15.150:6815","nonce":1557391161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1557391161},{"type":"v1","addr":"172.21.15.150:6812","nonce":1557391161}]},"public_addr":"172.21.15.150:6804/1557391161","cluster_addr":"172.21.15.150:6808/1557391161","heartbeat_back_addr":"172.21.15.150:6815/1557391161","heartbeat_front_addr":"172.21.15.150:6812/1557391161","state":["exists","up"]},{"osd":2,"uuid":"b14908e0-1927-45f6-868a-32938df0ef0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2990791785},{"type":"v1","addr":"172.21.15.150:6817","nonce":2990791785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2990791785},{"type":"v1","addr":"172.21.15.150:6819","nonce":2990791785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2990791785},{"type":"v1","addr":"172.21.15.150:6823","nonce":2990791785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2990791785},{"type":"v1","addr":"172.21.15.150:6821","nonce":2990791785}]},"public_addr":"172.21.15.150:6817/2990791785","cluster_addr":"172.21.15.150:6819/2990791785","heartbeat_back_addr":"172.21.15.150:6823/2990791785","heartbeat_front_addr":"172.21.15.150:6821/2990791785","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:52.931355+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:53.031762+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:52.998297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T21:19:01.825 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-04-23T21:19:01.825 INFO:tasks.ceph:Creating RBD pool 2024-04-23T21:19:01.826 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster cluster2 osd pool create rbd 8 2024-04-23T21:19:02.606 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'rbd' created 2024-04-23T21:19:02.622 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster cluster2 osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-23T21:19:03.607 INFO:teuthology.orchestra.run.smithi150.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-23T21:19:03.625 INFO:tasks.ceph:Starting mds daemons in cluster cluster2... 2024-04-23T21:19:03.625 INFO:tasks.ceph:Waiting until cluster2 daemons up and pgs clean... 2024-04-23T21:19:03.625 INFO:tasks.ceph.ceph_manager.cluster2:waiting for mgr available 2024-04-23T21:19:03.625 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 mgr dump --format=json 2024-04-23T21:19:03.626 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-23T21:19:03.959 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:19:03.971 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":5,"flags":0,"active_gid":4098,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":504844651},{"type":"v1","addr":"172.21.15.148:6827","nonce":504844651}]},"active_addr":"172.21.15.148:6827/504844651","active_change":"2024-04-23T21:18:50.512130+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":1288277618}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":4274377212}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":4007498785}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.148:0","nonce":1441098334}]}]} 2024-04-23T21:19:03.973 INFO:tasks.ceph.ceph_manager.cluster2:mgr available! 2024-04-23T21:19:03.973 INFO:tasks.ceph.ceph_manager.cluster2:waiting for all up 2024-04-23T21:19:03.973 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2024-04-23T21:19:04.256 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:19:04.256 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":14,"fsid":"04592e9d-e1b9-4919-8817-70bceab06a1f","created":"2024-04-23T21:18:47.128102+0000","modified":"2024-04-23T21:19:03.607713+0000","last_up_change":"2024-04-23T21:18:55.580476+0000","last_in_change":"2024-04-23T21:18:47.958645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T21:18:56.733503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T21:19:02.110649+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2321b5c-7aac-4fe7-9ec6-30b635f38c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4253773404},{"type":"v1","addr":"172.21.15.150:6801","nonce":4253773404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6803","nonce":4253773404},{"type":"v1","addr":"172.21.15.150:6805","nonce":4253773404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6811","nonce":4253773404},{"type":"v1","addr":"172.21.15.150:6813","nonce":4253773404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6807","nonce":4253773404},{"type":"v1","addr":"172.21.15.150:6809","nonce":4253773404}]},"public_addr":"172.21.15.150:6801/4253773404","cluster_addr":"172.21.15.150:6805/4253773404","heartbeat_back_addr":"172.21.15.150:6813/4253773404","heartbeat_front_addr":"172.21.15.150:6809/4253773404","state":["exists","up"]},{"osd":1,"uuid":"fad0d87f-9151-4aec-859f-1d499f45e002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1557391161},{"type":"v1","addr":"172.21.15.150:6804","nonce":1557391161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1557391161},{"type":"v1","addr":"172.21.15.150:6808","nonce":1557391161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1557391161},{"type":"v1","addr":"172.21.15.150:6815","nonce":1557391161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1557391161},{"type":"v1","addr":"172.21.15.150:6812","nonce":1557391161}]},"public_addr":"172.21.15.150:6804/1557391161","cluster_addr":"172.21.15.150:6808/1557391161","heartbeat_back_addr":"172.21.15.150:6815/1557391161","heartbeat_front_addr":"172.21.15.150:6812/1557391161","state":["exists","up"]},{"osd":2,"uuid":"b14908e0-1927-45f6-868a-32938df0ef0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2990791785},{"type":"v1","addr":"172.21.15.150:6817","nonce":2990791785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2990791785},{"type":"v1","addr":"172.21.15.150:6819","nonce":2990791785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2990791785},{"type":"v1","addr":"172.21.15.150:6823","nonce":2990791785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2990791785},{"type":"v1","addr":"172.21.15.150:6821","nonce":2990791785}]},"public_addr":"172.21.15.150:6817/2990791785","cluster_addr":"172.21.15.150:6819/2990791785","heartbeat_back_addr":"172.21.15.150:6823/2990791785","heartbeat_front_addr":"172.21.15.150:6821/2990791785","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:52.931355+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:53.031762+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:52.998297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T21:19:04.269 INFO:tasks.ceph.ceph_manager.cluster2:all up! 2024-04-23T21:19:04.269 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2024-04-23T21:19:04.556 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:19:04.556 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":14,"fsid":"04592e9d-e1b9-4919-8817-70bceab06a1f","created":"2024-04-23T21:18:47.128102+0000","modified":"2024-04-23T21:19:03.607713+0000","last_up_change":"2024-04-23T21:18:55.580476+0000","last_in_change":"2024-04-23T21:18:47.958645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T21:18:56.733503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T21:19:02.110649+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2321b5c-7aac-4fe7-9ec6-30b635f38c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4253773404},{"type":"v1","addr":"172.21.15.150:6801","nonce":4253773404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6803","nonce":4253773404},{"type":"v1","addr":"172.21.15.150:6805","nonce":4253773404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6811","nonce":4253773404},{"type":"v1","addr":"172.21.15.150:6813","nonce":4253773404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6807","nonce":4253773404},{"type":"v1","addr":"172.21.15.150:6809","nonce":4253773404}]},"public_addr":"172.21.15.150:6801/4253773404","cluster_addr":"172.21.15.150:6805/4253773404","heartbeat_back_addr":"172.21.15.150:6813/4253773404","heartbeat_front_addr":"172.21.15.150:6809/4253773404","state":["exists","up"]},{"osd":1,"uuid":"fad0d87f-9151-4aec-859f-1d499f45e002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1557391161},{"type":"v1","addr":"172.21.15.150:6804","nonce":1557391161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1557391161},{"type":"v1","addr":"172.21.15.150:6808","nonce":1557391161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1557391161},{"type":"v1","addr":"172.21.15.150:6815","nonce":1557391161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1557391161},{"type":"v1","addr":"172.21.15.150:6812","nonce":1557391161}]},"public_addr":"172.21.15.150:6804/1557391161","cluster_addr":"172.21.15.150:6808/1557391161","heartbeat_back_addr":"172.21.15.150:6815/1557391161","heartbeat_front_addr":"172.21.15.150:6812/1557391161","state":["exists","up"]},{"osd":2,"uuid":"b14908e0-1927-45f6-868a-32938df0ef0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2990791785},{"type":"v1","addr":"172.21.15.150:6817","nonce":2990791785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2990791785},{"type":"v1","addr":"172.21.15.150:6819","nonce":2990791785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2990791785},{"type":"v1","addr":"172.21.15.150:6823","nonce":2990791785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2990791785},{"type":"v1","addr":"172.21.15.150:6821","nonce":2990791785}]},"public_addr":"172.21.15.150:6817/2990791785","cluster_addr":"172.21.15.150:6819/2990791785","heartbeat_back_addr":"172.21.15.150:6823/2990791785","heartbeat_front_addr":"172.21.15.150:6821/2990791785","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:52.931355+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:53.031762+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:52.998297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T21:19:04.569 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.0 flush_pg_stats 2024-04-23T21:19:04.570 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.1 flush_pg_stats 2024-04-23T21:19:04.570 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.2 flush_pg_stats 2024-04-23T21:19:04.707 INFO:teuthology.orchestra.run.smithi150.stdout:34359738374 2024-04-23T21:19:04.708 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.2 2024-04-23T21:19:04.724 INFO:teuthology.orchestra.run.smithi150.stdout:34359738373 2024-04-23T21:19:04.724 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2024-04-23T21:19:04.725 INFO:teuthology.orchestra.run.smithi150.stdout:34359738374 2024-04-23T21:19:04.725 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.1 2024-04-23T21:19:05.013 INFO:teuthology.orchestra.run.smithi150.stdout:34359738373 2024-04-23T21:19:05.026 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738374 got 34359738373 for osd.2 2024-04-23T21:19:05.066 INFO:teuthology.orchestra.run.smithi150.stdout:34359738373 2024-04-23T21:19:05.068 INFO:teuthology.orchestra.run.smithi150.stdout:34359738372 2024-04-23T21:19:05.080 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738374 got 34359738373 for osd.1 2024-04-23T21:19:05.081 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738373 got 34359738372 for osd.0 2024-04-23T21:19:06.027 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.2 2024-04-23T21:19:06.081 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.1 2024-04-23T21:19:06.082 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2024-04-23T21:19:06.350 INFO:teuthology.orchestra.run.smithi150.stdout:34359738373 2024-04-23T21:19:06.363 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738374 got 34359738373 for osd.2 2024-04-23T21:19:06.387 INFO:teuthology.orchestra.run.smithi150.stdout:34359738372 2024-04-23T21:19:06.394 INFO:teuthology.orchestra.run.smithi150.stdout:34359738373 2024-04-23T21:19:06.400 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738373 got 34359738372 for osd.0 2024-04-23T21:19:06.408 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738374 got 34359738373 for osd.1 2024-04-23T21:19:07.363 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.2 2024-04-23T21:19:07.401 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2024-04-23T21:19:07.408 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.1 2024-04-23T21:19:07.678 INFO:teuthology.orchestra.run.smithi150.stdout:34359738374 2024-04-23T21:19:07.691 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738374 got 34359738374 for osd.2 2024-04-23T21:19:07.691 DEBUG:teuthology.parallel:result is None 2024-04-23T21:19:07.715 INFO:teuthology.orchestra.run.smithi150.stdout:34359738374 2024-04-23T21:19:07.722 INFO:teuthology.orchestra.run.smithi150.stdout:34359738374 2024-04-23T21:19:07.729 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738373 got 34359738374 for osd.0 2024-04-23T21:19:07.730 DEBUG:teuthology.parallel:result is None 2024-04-23T21:19:07.735 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738374 got 34359738374 for osd.1 2024-04-23T21:19:07.736 DEBUG:teuthology.parallel:result is None 2024-04-23T21:19:07.736 INFO:tasks.ceph.ceph_manager.cluster2:waiting for clean 2024-04-23T21:19:07.736 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-04-23T21:19:08.061 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:19:08.061 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-23T21:19:08.073 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-23T21:19:06.518828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81552,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283033968,"statfs":{"total":289910292480,"available":289826783232,"internally_reserved":0,"allocated":491520,"data_stored":1285482,"data_compressed":5808,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":4784,"internal_metadata":82963792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000691"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:19:03.618557+0000","last_change":"2024-04-23T21:19:03.617238+0000","last_active":"2024-04-23T21:19:03.618557+0000","last_peered":"2024-04-23T21:19:03.618557+0000","last_clean":"2024-04-23T21:19:03.618557+0000","last_became_active":"2024-04-23T21:19:03.616645+0000","last_became_peered":"2024-04-23T21:19:03.616645+0000","last_unstale":"2024-04-23T21:19:03.618557+0000","last_undegraded":"2024-04-23T21:19:03.618557+0000","last_fullsized":"2024-04-23T21:19:03.618557+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:48:39.006728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:19:03.618584+0000","last_change":"2024-04-23T21:19:03.617657+0000","last_active":"2024-04-23T21:19:03.618584+0000","last_peered":"2024-04-23T21:19:03.618584+0000","last_clean":"2024-04-23T21:19:03.618584+0000","last_became_active":"2024-04-23T21:19:03.616675+0000","last_became_peered":"2024-04-23T21:19:03.616675+0000","last_unstale":"2024-04-23T21:19:03.618584+0000","last_undegraded":"2024-04-23T21:19:03.618584+0000","last_fullsized":"2024-04-23T21:19:03.618584+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:25:10.153387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:19:03.619114+0000","last_change":"2024-04-23T21:19:03.617922+0000","last_active":"2024-04-23T21:19:03.619114+0000","last_peered":"2024-04-23T21:19:03.619114+0000","last_clean":"2024-04-23T21:19:03.619114+0000","last_became_active":"2024-04-23T21:19:03.616607+0000","last_became_peered":"2024-04-23T21:19:03.616607+0000","last_unstale":"2024-04-23T21:19:03.619114+0000","last_undegraded":"2024-04-23T21:19:03.619114+0000","last_fullsized":"2024-04-23T21:19:03.619114+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:19:45.986660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:19:03.618518+0000","last_change":"2024-04-23T21:19:03.616894+0000","last_active":"2024-04-23T21:19:03.618518+0000","last_peered":"2024-04-23T21:19:03.618518+0000","last_clean":"2024-04-23T21:19:03.618518+0000","last_became_active":"2024-04-23T21:19:03.616601+0000","last_became_peered":"2024-04-23T21:19:03.616601+0000","last_unstale":"2024-04-23T21:19:03.618518+0000","last_undegraded":"2024-04-23T21:19:03.618518+0000","last_fullsized":"2024-04-23T21:19:03.618518+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:49:58.372259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T21:19:04.617843+0000","last_change":"2024-04-23T21:19:03.615613+0000","last_active":"2024-04-23T21:19:04.617843+0000","last_peered":"2024-04-23T21:19:04.617843+0000","last_clean":"2024-04-23T21:19:04.617843+0000","last_became_active":"2024-04-23T21:19:03.615283+0000","last_became_peered":"2024-04-23T21:19:03.615283+0000","last_unstale":"2024-04-23T21:19:04.617843+0000","last_undegraded":"2024-04-23T21:19:04.617843+0000","last_fullsized":"2024-04-23T21:19:04.617843+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:14:30.533352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:19:03.619131+0000","last_change":"2024-04-23T21:19:03.617229+0000","last_active":"2024-04-23T21:19:03.619131+0000","last_peered":"2024-04-23T21:19:03.619131+0000","last_clean":"2024-04-23T21:19:03.619131+0000","last_became_active":"2024-04-23T21:19:03.616884+0000","last_became_peered":"2024-04-23T21:19:03.616884+0000","last_unstale":"2024-04-23T21:19:03.619131+0000","last_undegraded":"2024-04-23T21:19:03.619131+0000","last_fullsized":"2024-04-23T21:19:03.619131+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:23.988072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:19:03.619157+0000","last_change":"2024-04-23T21:19:03.617745+0000","last_active":"2024-04-23T21:19:03.619157+0000","last_peered":"2024-04-23T21:19:03.619157+0000","last_clean":"2024-04-23T21:19:03.619157+0000","last_became_active":"2024-04-23T21:19:03.617420+0000","last_became_peered":"2024-04-23T21:19:03.617420+0000","last_unstale":"2024-04-23T21:19:03.619157+0000","last_undegraded":"2024-04-23T21:19:03.619157+0000","last_fullsized":"2024-04-23T21:19:03.619157+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:17:16.637931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:19:03.618483+0000","last_change":"2024-04-23T21:19:03.616216+0000","last_active":"2024-04-23T21:19:03.618483+0000","last_peered":"2024-04-23T21:19:03.618483+0000","last_clean":"2024-04-23T21:19:03.618483+0000","last_became_active":"2024-04-23T21:19:03.615832+0000","last_became_peered":"2024-04-23T21:19:03.615832+0000","last_unstale":"2024-04-23T21:19:03.618483+0000","last_undegraded":"2024-04-23T21:19:03.618483+0000","last_fullsized":"2024-04-23T21:19:03.618483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:57:19.503329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":143,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:19:03.610846+0000","last_change":"2024-04-23T21:18:58.602249+0000","last_active":"2024-04-23T21:19:03.610846+0000","last_peered":"2024-04-23T21:19:03.610846+0000","last_clean":"2024-04-23T21:19:03.610846+0000","last_became_active":"2024-04-23T21:18:58.600198+0000","last_became_peered":"2024-04-23T21:18:58.600198+0000","last_unstale":"2024-04-23T21:19:03.610846+0000","last_undegraded":"2024-04-23T21:19:03.610846+0000","last_fullsized":"2024-04-23T21:19:03.610846+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:57.592214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:57.592214+0000","last_clean_scrub_stamp":"2024-04-23T21:18:57.592214+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:36:39.822147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":5808,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":624124,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27140,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344700,"statfs":{"total":96636764160,"available":96608972800,"internally_reserved":0,"allocated":122880,"data_stored":33756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":188416,"data_stored":627602,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T21:19:08.074 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-04-23T21:19:08.352 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:19:08.418 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-23T21:19:08.420 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-04-23T21:19:06.518828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81552,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283033968,"statfs":{"total":289910292480,"available":289826783232,"internally_reserved":0,"allocated":491520,"data_stored":1285482,"data_compressed":5808,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":4784,"internal_metadata":82963792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000691"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:19:03.618557+0000","last_change":"2024-04-23T21:19:03.617238+0000","last_active":"2024-04-23T21:19:03.618557+0000","last_peered":"2024-04-23T21:19:03.618557+0000","last_clean":"2024-04-23T21:19:03.618557+0000","last_became_active":"2024-04-23T21:19:03.616645+0000","last_became_peered":"2024-04-23T21:19:03.616645+0000","last_unstale":"2024-04-23T21:19:03.618557+0000","last_undegraded":"2024-04-23T21:19:03.618557+0000","last_fullsized":"2024-04-23T21:19:03.618557+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:48:39.006728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:19:03.618584+0000","last_change":"2024-04-23T21:19:03.617657+0000","last_active":"2024-04-23T21:19:03.618584+0000","last_peered":"2024-04-23T21:19:03.618584+0000","last_clean":"2024-04-23T21:19:03.618584+0000","last_became_active":"2024-04-23T21:19:03.616675+0000","last_became_peered":"2024-04-23T21:19:03.616675+0000","last_unstale":"2024-04-23T21:19:03.618584+0000","last_undegraded":"2024-04-23T21:19:03.618584+0000","last_fullsized":"2024-04-23T21:19:03.618584+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:25:10.153387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:19:03.619114+0000","last_change":"2024-04-23T21:19:03.617922+0000","last_active":"2024-04-23T21:19:03.619114+0000","last_peered":"2024-04-23T21:19:03.619114+0000","last_clean":"2024-04-23T21:19:03.619114+0000","last_became_active":"2024-04-23T21:19:03.616607+0000","last_became_peered":"2024-04-23T21:19:03.616607+0000","last_unstale":"2024-04-23T21:19:03.619114+0000","last_undegraded":"2024-04-23T21:19:03.619114+0000","last_fullsized":"2024-04-23T21:19:03.619114+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:19:45.986660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:19:03.618518+0000","last_change":"2024-04-23T21:19:03.616894+0000","last_active":"2024-04-23T21:19:03.618518+0000","last_peered":"2024-04-23T21:19:03.618518+0000","last_clean":"2024-04-23T21:19:03.618518+0000","last_became_active":"2024-04-23T21:19:03.616601+0000","last_became_peered":"2024-04-23T21:19:03.616601+0000","last_unstale":"2024-04-23T21:19:03.618518+0000","last_undegraded":"2024-04-23T21:19:03.618518+0000","last_fullsized":"2024-04-23T21:19:03.618518+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:49:58.372259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-23T21:19:04.617843+0000","last_change":"2024-04-23T21:19:03.615613+0000","last_active":"2024-04-23T21:19:04.617843+0000","last_peered":"2024-04-23T21:19:04.617843+0000","last_clean":"2024-04-23T21:19:04.617843+0000","last_became_active":"2024-04-23T21:19:03.615283+0000","last_became_peered":"2024-04-23T21:19:03.615283+0000","last_unstale":"2024-04-23T21:19:04.617843+0000","last_undegraded":"2024-04-23T21:19:04.617843+0000","last_fullsized":"2024-04-23T21:19:04.617843+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:14:30.533352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:19:03.619131+0000","last_change":"2024-04-23T21:19:03.617229+0000","last_active":"2024-04-23T21:19:03.619131+0000","last_peered":"2024-04-23T21:19:03.619131+0000","last_clean":"2024-04-23T21:19:03.619131+0000","last_became_active":"2024-04-23T21:19:03.616884+0000","last_became_peered":"2024-04-23T21:19:03.616884+0000","last_unstale":"2024-04-23T21:19:03.619131+0000","last_undegraded":"2024-04-23T21:19:03.619131+0000","last_fullsized":"2024-04-23T21:19:03.619131+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:35:23.988072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:19:03.619157+0000","last_change":"2024-04-23T21:19:03.617745+0000","last_active":"2024-04-23T21:19:03.619157+0000","last_peered":"2024-04-23T21:19:03.619157+0000","last_clean":"2024-04-23T21:19:03.619157+0000","last_became_active":"2024-04-23T21:19:03.617420+0000","last_became_peered":"2024-04-23T21:19:03.617420+0000","last_unstale":"2024-04-23T21:19:03.619157+0000","last_undegraded":"2024-04-23T21:19:03.619157+0000","last_fullsized":"2024-04-23T21:19:03.619157+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:17:16.637931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:19:03.618483+0000","last_change":"2024-04-23T21:19:03.616216+0000","last_active":"2024-04-23T21:19:03.618483+0000","last_peered":"2024-04-23T21:19:03.618483+0000","last_clean":"2024-04-23T21:19:03.618483+0000","last_became_active":"2024-04-23T21:19:03.615832+0000","last_became_peered":"2024-04-23T21:19:03.615832+0000","last_unstale":"2024-04-23T21:19:03.618483+0000","last_undegraded":"2024-04-23T21:19:03.618483+0000","last_fullsized":"2024-04-23T21:19:03.618483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:57:19.503329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":143,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-04-23T21:19:03.610846+0000","last_change":"2024-04-23T21:18:58.602249+0000","last_active":"2024-04-23T21:19:03.610846+0000","last_peered":"2024-04-23T21:19:03.610846+0000","last_clean":"2024-04-23T21:19:03.610846+0000","last_became_active":"2024-04-23T21:18:58.600198+0000","last_became_peered":"2024-04-23T21:18:58.600198+0000","last_unstale":"2024-04-23T21:19:03.610846+0000","last_undegraded":"2024-04-23T21:19:03.610846+0000","last_fullsized":"2024-04-23T21:19:03.610846+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:57.592214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:57.592214+0000","last_clean_scrub_stamp":"2024-04-23T21:18:57.592214+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:36:39.822147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":5808,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":624124,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27140,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344700,"statfs":{"total":96636764160,"available":96608972800,"internally_reserved":0,"allocated":122880,"data_stored":33756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":188416,"data_stored":627602,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T21:19:08.421 INFO:tasks.ceph.ceph_manager.cluster2:clean! 2024-04-23T21:19:08.421 INFO:tasks.ceph:Waiting until ceph cluster cluster2 is healthy... 2024-04-23T21:19:08.421 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy 2024-04-23T21:19:08.421 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 health --format=json 2024-04-23T21:19:08.742 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:19:08.742 INFO:teuthology.orchestra.run.smithi150.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T21:19:08.755 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy done 2024-04-23T21:19:08.755 INFO:teuthology.run_tasks:Running task exec... 2024-04-23T21:19:08.805 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T21:19:08.805 INFO:teuthology.task.exec:Running commands on role cluster1.client.mirror.0 host ubuntu@smithi150.front.sepia.ceph.com 2024-04-23T21:19:08.806 DEBUG:teuthology.orchestra.run.smithi150:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror mon '"'"'profile rbd-mirror-peer'"'"' osd '"'"'profile rbd'"'"'' 2024-04-23T21:19:09.168 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror] 2024-04-23T21:19:09.168 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCFJShmRR5VLRAAMOVjiRzcJfepgCt6uJB2Xw== 2024-04-23T21:19:09.168 INFO:teuthology.orchestra.run.smithi150.stdout: caps mon = "profile rbd-mirror-peer" 2024-04-23T21:19:09.169 INFO:teuthology.orchestra.run.smithi150.stdout: caps osd = "profile rbd" 2024-04-23T21:19:09.169 INFO:teuthology.orchestra.run.smithi150.stderr:updated caps for client.mirror 2024-04-23T21:19:09.182 DEBUG:teuthology.orchestra.run.smithi150:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.0 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2024-04-23T21:19:09.546 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.0] 2024-04-23T21:19:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCFJShmJcaNMBAA6LzBW8HDCgEeTNXz25ePzg== 2024-04-23T21:19:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: caps mon = "profile rbd-mirror" 2024-04-23T21:19:09.547 INFO:teuthology.orchestra.run.smithi150.stdout: caps osd = "profile rbd" 2024-04-23T21:19:09.547 INFO:teuthology.orchestra.run.smithi150.stderr:updated caps for client.mirror.0 2024-04-23T21:19:09.560 DEBUG:teuthology.orchestra.run.smithi150:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.1 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2024-04-23T21:19:09.924 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.1] 2024-04-23T21:19:09.924 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCFJShmeUUNNhAAe7juz4n7Jvjj/OkRxPBBnQ== 2024-04-23T21:19:09.924 INFO:teuthology.orchestra.run.smithi150.stdout: caps mon = "profile rbd-mirror" 2024-04-23T21:19:09.924 INFO:teuthology.orchestra.run.smithi150.stdout: caps osd = "profile rbd" 2024-04-23T21:19:09.924 INFO:teuthology.orchestra.run.smithi150.stderr:updated caps for client.mirror.1 2024-04-23T21:19:09.938 DEBUG:teuthology.orchestra.run.smithi150:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.2 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2024-04-23T21:19:10.304 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.2] 2024-04-23T21:19:10.304 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCFJShmGoN1OxAAaYY2nUFrzt1Z+CpAr22zWQ== 2024-04-23T21:19:10.304 INFO:teuthology.orchestra.run.smithi150.stdout: caps mon = "profile rbd-mirror" 2024-04-23T21:19:10.305 INFO:teuthology.orchestra.run.smithi150.stdout: caps osd = "profile rbd" 2024-04-23T21:19:10.305 INFO:teuthology.orchestra.run.smithi150.stderr:updated caps for client.mirror.2 2024-04-23T21:19:10.319 DEBUG:teuthology.orchestra.run.smithi150:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.3 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2024-04-23T21:19:10.680 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.3] 2024-04-23T21:19:10.680 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCGJShm79xEBRAAstBk8oqga/923bSRrLvaog== 2024-04-23T21:19:10.680 INFO:teuthology.orchestra.run.smithi150.stdout: caps mon = "profile rbd-mirror" 2024-04-23T21:19:10.680 INFO:teuthology.orchestra.run.smithi150.stdout: caps osd = "profile rbd" 2024-04-23T21:19:10.680 INFO:teuthology.orchestra.run.smithi150.stderr:updated caps for client.mirror.3 2024-04-23T21:19:10.693 DEBUG:teuthology.orchestra.run.smithi150:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror mon '"'"'profile rbd-mirror-peer'"'"' osd '"'"'profile rbd'"'"'' 2024-04-23T21:19:11.054 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror] 2024-04-23T21:19:11.054 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCpJShmsJ2dIBAA2A53DeXSvfNJKprwNBgHaw== 2024-04-23T21:19:11.054 INFO:teuthology.orchestra.run.smithi150.stdout: caps mon = "profile rbd-mirror-peer" 2024-04-23T21:19:11.055 INFO:teuthology.orchestra.run.smithi150.stdout: caps osd = "profile rbd" 2024-04-23T21:19:11.055 INFO:teuthology.orchestra.run.smithi150.stderr:updated caps for client.mirror 2024-04-23T21:19:11.068 DEBUG:teuthology.orchestra.run.smithi150:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.0 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2024-04-23T21:19:11.432 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.0] 2024-04-23T21:19:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCpJShmnxkBJhAAmue1/B93gFR+6eVYamaVHg== 2024-04-23T21:19:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: caps mon = "profile rbd-mirror" 2024-04-23T21:19:11.432 INFO:teuthology.orchestra.run.smithi150.stdout: caps osd = "profile rbd" 2024-04-23T21:19:11.432 INFO:teuthology.orchestra.run.smithi150.stderr:updated caps for client.mirror.0 2024-04-23T21:19:11.445 DEBUG:teuthology.orchestra.run.smithi150:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.1 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2024-04-23T21:19:11.807 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.1] 2024-04-23T21:19:11.807 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCpJShm08xrKxAA6QtxWMEHkqJFJ6lp+Ho/KA== 2024-04-23T21:19:11.807 INFO:teuthology.orchestra.run.smithi150.stdout: caps mon = "profile rbd-mirror" 2024-04-23T21:19:11.807 INFO:teuthology.orchestra.run.smithi150.stdout: caps osd = "profile rbd" 2024-04-23T21:19:11.807 INFO:teuthology.orchestra.run.smithi150.stderr:updated caps for client.mirror.1 2024-04-23T21:19:11.820 DEBUG:teuthology.orchestra.run.smithi150:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.2 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2024-04-23T21:19:12.182 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.2] 2024-04-23T21:19:12.182 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCpJShmytHLMBAAW1Urc6BwirEM68rUs2Ozkg== 2024-04-23T21:19:12.182 INFO:teuthology.orchestra.run.smithi150.stdout: caps mon = "profile rbd-mirror" 2024-04-23T21:19:12.182 INFO:teuthology.orchestra.run.smithi150.stdout: caps osd = "profile rbd" 2024-04-23T21:19:12.182 INFO:teuthology.orchestra.run.smithi150.stderr:updated caps for client.mirror.2 2024-04-23T21:19:12.195 DEBUG:teuthology.orchestra.run.smithi150:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.3 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2024-04-23T21:19:12.563 INFO:teuthology.orchestra.run.smithi150.stdout:[client.mirror.3] 2024-04-23T21:19:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQCpJShmNnQ3NhAAvI3BiwXT15yzYV2kixHEuQ== 2024-04-23T21:19:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: caps mon = "profile rbd-mirror" 2024-04-23T21:19:12.563 INFO:teuthology.orchestra.run.smithi150.stdout: caps osd = "profile rbd" 2024-04-23T21:19:12.563 INFO:teuthology.orchestra.run.smithi150.stderr:updated caps for client.mirror.3 2024-04-23T21:19:12.576 INFO:teuthology.run_tasks:Running task rbd-mirror... 2024-04-23T21:19:12.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2024-04-23T21:19:12.587 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2024-04-23T21:19:12.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2024-04-23T21:19:12.589 INFO:teuthology.run_tasks:Running task rbd-mirror... 2024-04-23T21:19:12.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2024-04-23T21:19:12.601 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2024-04-23T21:19:12.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2024-04-23T21:19:12.604 INFO:teuthology.run_tasks:Running task rbd-mirror... 2024-04-23T21:19:12.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2024-04-23T21:19:12.612 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2024-04-23T21:19:12.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2024-04-23T21:19:12.614 INFO:teuthology.run_tasks:Running task rbd-mirror... 2024-04-23T21:19:12.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2024-04-23T21:19:12.623 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2024-04-23T21:19:12.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2024-04-23T21:19:12.625 INFO:teuthology.run_tasks:Running task rbd-mirror... 2024-04-23T21:19:12.635 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2024-04-23T21:19:12.635 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2024-04-23T21:19:12.637 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2024-04-23T21:19:12.637 INFO:teuthology.run_tasks:Running task rbd-mirror... 2024-04-23T21:19:12.646 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2024-04-23T21:19:12.646 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2024-04-23T21:19:12.649 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2024-04-23T21:19:12.649 INFO:teuthology.run_tasks:Running task rbd-mirror... 2024-04-23T21:19:12.659 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2024-04-23T21:19:12.659 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2024-04-23T21:19:12.661 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2024-04-23T21:19:12.661 INFO:teuthology.run_tasks:Running task rbd-mirror... 2024-04-23T21:19:12.670 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2024-04-23T21:19:12.670 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2024-04-23T21:19:12.673 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2024-04-23T21:19:12.673 INFO:teuthology.run_tasks:Running task rbd-mirror-thrash... 2024-04-23T21:19:12.684 INFO:tasks.rbd_mirror_thrash:rbd_mirror_thrash using random seed: 1713907152 2024-04-23T21:19:12.684 DEBUG:tasks.rbd_mirror_thrash:Yielding 2024-04-23T21:19:12.684 INFO:teuthology.run_tasks:Running task rbd-mirror-thrash... 2024-04-23T21:19:12.692 INFO:tasks.rbd_mirror_thrash:rbd_mirror_thrash using random seed: 1713907152 2024-04-23T21:19:12.692 DEBUG:tasks.rbd_mirror_thrash:Yielding 2024-04-23T21:19:12.693 INFO:teuthology.run_tasks:Running task workunit... 2024-04-23T21:19:12.707 INFO:tasks.workunit:Pulling workunits from ref c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f 2024-04-23T21:19:12.707 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-23T21:19:12.707 DEBUG:teuthology.orchestra.run.smithi150:> stat -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2024-04-23T21:19:12.707 INFO:tasks.rbd_mirror_thrash:starting thrash for cluster cluster1 2024-04-23T21:19:12.707 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2024-04-23T21:19:12.708 INFO:tasks.rbd_mirror_thrash:starting thrash for cluster cluster2 2024-04-23T21:19:12.708 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2024-04-23T21:19:12.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T21:19:12.728 INFO:teuthology.orchestra.run.smithi150.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.cluster1.mirror': No such file or directory 2024-04-23T21:19:12.728 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2024-04-23T21:19:12.783 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.cluster1.mirror 2024-04-23T21:19:12.783 DEBUG:teuthology.orchestra.run.smithi150:> cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror && mkdir -- client.mirror 2024-04-23T21:19:12.837 INFO:tasks.workunit:timeout=6h 2024-04-23T21:19:12.838 INFO:tasks.workunit:cleanup=True 2024-04-23T21:19:12.838 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/clone.cluster1.client.mirror && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.cluster1.client.mirror && cd /home/ubuntu/cephtest/clone.cluster1.client.mirror && git checkout c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f 2024-04-23T21:19:12.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Cloning into '/home/ubuntu/cephtest/clone.cluster1.client.mirror'... 2024-04-23T21:20:23.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Note: switching to 'c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f'. 2024-04-23T21:20:23.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: 2024-04-23T21:20:23.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-23T21:20:23.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-23T21:20:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:state without impacting any branches by switching back to a branch. 2024-04-23T21:20:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: 2024-04-23T21:20:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-23T21:20:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-23T21:20:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: 2024-04-23T21:20:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: git switch -c 2024-04-23T21:20:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: 2024-04-23T21:20:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Or undo this operation with: 2024-04-23T21:20:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: 2024-04-23T21:20:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: git switch - 2024-04-23T21:20:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: 2024-04-23T21:20:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-23T21:20:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: 2024-04-23T21:20:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:HEAD is now at c37f63b3705 Merge pull request #57053 from idryomov/wip-labeler-rbd 2024-04-23T21:20:23.958 DEBUG:teuthology.orchestra.run.smithi150:> cd -- /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror 2024-04-23T21:20:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-23T21:20:24.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2024-04-23T21:20:24.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-23T21:20:24.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-23T21:20:24.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-23T21:20:24.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2024-04-23T21:20:24.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2024-04-23T21:20:24.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-23T21:20:25.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2024-04-23T21:20:25.005 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-23T21:20:25.005 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror of=/dev/stdout 2024-04-23T21:20:25.020 INFO:tasks.workunit:Running workunits matching rbd/rbd_mirror_stress.sh on cluster1.client.mirror... 2024-04-23T21:20:25.021 INFO:tasks.workunit:Running workunit rbd/rbd_mirror_stress.sh... 2024-04-23T21:20:25.022 DEBUG:teuthology.orchestra.run.smithi150:workunit test rbd/rbd_mirror_stress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c37f63b3705f40f1bfcd7d8aeb311499c65cfa1f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS='' MIRROR_POOL_MODE=pool RBD_MIRROR_INSTANCES=4 RBD_MIRROR_MODE=journal RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2024-04-23T21:20:25.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ IMAGE_COUNT=50 2024-04-23T21:20:25.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ export LOCKDEP=0 2024-04-23T21:20:25.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ LOCKDEP=0 2024-04-23T21:20:25.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2024-04-23T21:20:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ . /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_helpers.sh 2024-04-23T21:20:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ type xmlstarlet 2024-04-23T21:20:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ XMLSTARLET=xmlstarlet 2024-04-23T21:20:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ RBD_MIRROR_INSTANCES=4 2024-04-23T21:20:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CLUSTER1=cluster1 2024-04-23T21:20:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CLUSTER2=cluster2 2024-04-23T21:20:25.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ PEER_CLUSTER_SUFFIX= 2024-04-23T21:20:25.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ POOL=mirror 2024-04-23T21:20:25.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ PARENT_POOL=mirror_parent 2024-04-23T21:20:25.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ NS1=ns1 2024-04-23T21:20:25.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ NS2=ns2 2024-04-23T21:20:25.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ TEMPDIR= 2024-04-23T21:20:25.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ID=mirror 2024-04-23T21:20:25.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ RBD_IMAGE_FEATURES=layering,exclusive-lock,journaling 2024-04-23T21:20:25.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ MIRROR_USER_ID_PREFIX=mirror. 2024-04-23T21:20:25.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ RBD_MIRROR_MODE=journal 2024-04-23T21:20:25.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ MIRROR_POOL_MODE=pool 2024-04-23T21:20:25.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' journal = snapshot ']' 2024-04-23T21:20:25.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ export 'CEPH_ARGS=--id mirror' 2024-04-23T21:20:25.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS='--id mirror' 2024-04-23T21:20:25.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ LAST_MIRROR_INSTANCE=3 2024-04-23T21:20:25.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2024-04-23T21:20:25.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ readlink -f /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/../../../src 2024-04-23T21:20:25.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror/src 2024-04-23T21:20:25.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_BIN=. 2024-04-23T21:20:25.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_SRC=. 2024-04-23T21:20:25.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' -e CMakeCache.txt ']' 2024-04-23T21:20:25.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 -gt 0 ']' 2024-04-23T21:20:25.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ setup 2024-04-23T21:20:25.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local c 2024-04-23T21:20:25.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ trap 'cleanup $?' INT TERM EXIT 2024-04-23T21:20:25.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ setup_tempdir 2024-04-23T21:20:25.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:20:25.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ mktemp -d 2024-04-23T21:20:25.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ TEMPDIR=/tmp/tmp.8Tyc9aSFBO 2024-04-23T21:20:25.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -z 1 ']' 2024-04-23T21:20:25.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ setup_pools cluster1 cluster2 2024-04-23T21:20:25.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:20:25.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local remote_cluster=cluster2 2024-04-23T21:20:25.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mon_map_file 2024-04-23T21:20:25.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mon_addr 2024-04-23T21:20:25.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local admin_key_file 2024-04-23T21:20:25.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local uuid 2024-04-23T21:20:25.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ CEPH_ARGS= 2024-04-23T21:20:25.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --cluster cluster1 osd pool create mirror 64 64 2024-04-23T21:20:26.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:pool 'mirror' already exists 2024-04-23T21:20:26.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ CEPH_ARGS= 2024-04-23T21:20:26.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --cluster cluster1 osd pool create mirror_parent 64 64 2024-04-23T21:20:27.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:pool 'mirror_parent' already exists 2024-04-23T21:20:27.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ CEPH_ARGS= 2024-04-23T21:20:27.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 pool init mirror 2024-04-23T21:20:30.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ CEPH_ARGS= 2024-04-23T21:20:30.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 pool init mirror_parent 2024-04-23T21:20:33.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:20:33.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ CEPH_ARGS= 2024-04-23T21:20:33.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 mirror pool enable --site-name cluster1 mirror pool 2024-04-23T21:20:33.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 mirror pool enable mirror_parent image 2024-04-23T21:20:33.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 namespace create mirror/ns1 2024-04-23T21:20:33.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 namespace create mirror/ns2 2024-04-23T21:20:33.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns1 pool 2024-04-23T21:20:33.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns2 image 2024-04-23T21:20:33.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -z ']' 2024-04-23T21:20:33.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -z ']' 2024-04-23T21:20:33.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ peer_add cluster1 mirror cluster2 2024-04-23T21:20:33.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:20:33.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:33.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:20:33.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:33.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local client_cluster=cluster2 2024-04-23T21:20:33.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:33.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local remote_cluster=cluster2 2024-04-23T21:20:33.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local uuid_var_name 2024-04-23T21:20:33.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:20:33.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local error_code 2024-04-23T21:20:33.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local peer_uuid 2024-04-23T21:20:33.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 16 32 2024-04-23T21:20:33.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:20:33.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror pool peer add mirror cluster2 2024-04-23T21:20:33.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ peer_uuid=38aea5d5-3d8e-44a1-bf69-12083687b763 2024-04-23T21:20:33.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=0 2024-04-23T21:20:33.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:20:33.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 -eq 17 ']' 2024-04-23T21:20:33.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 0 -eq 0 2024-04-23T21:20:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:20:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:20:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ peer_add cluster1 mirror_parent cluster2 2024-04-23T21:20:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:20:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror_parent 2024-04-23T21:20:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local client_cluster=cluster2 2024-04-23T21:20:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local remote_cluster=cluster2 2024-04-23T21:20:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local uuid_var_name 2024-04-23T21:20:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:20:33.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local error_code 2024-04-23T21:20:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local peer_uuid 2024-04-23T21:20:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 16 32 2024-04-23T21:20:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:20:33.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror pool peer add mirror_parent cluster2 2024-04-23T21:20:33.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ peer_uuid=e8cfaf48-e208-4de2-a791-8072a511293d 2024-04-23T21:20:33.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=0 2024-04-23T21:20:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:20:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 -eq 17 ']' 2024-04-23T21:20:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 0 -eq 0 2024-04-23T21:20:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:20:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:20:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ setup_pools cluster2 cluster1 2024-04-23T21:20:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:20:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local remote_cluster=cluster1 2024-04-23T21:20:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mon_map_file 2024-04-23T21:20:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mon_addr 2024-04-23T21:20:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local admin_key_file 2024-04-23T21:20:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local uuid 2024-04-23T21:20:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ CEPH_ARGS= 2024-04-23T21:20:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --cluster cluster2 osd pool create mirror 64 64 2024-04-23T21:20:33.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:pool 'mirror' already exists 2024-04-23T21:20:33.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ CEPH_ARGS= 2024-04-23T21:20:33.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --cluster cluster2 osd pool create mirror_parent 64 64 2024-04-23T21:20:34.709 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2024-04-23T21:20:34.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2024-04-23T21:20:34.710 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2024-04-23T21:20:34.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:20:34.711+0000 7f29c25e4640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 35384) UID: 1000 2024-04-23T21:20:34.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:pool 'mirror_parent' already exists 2024-04-23T21:20:34.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ CEPH_ARGS= 2024-04-23T21:20:34.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 pool init mirror 2024-04-23T21:20:35.711 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2024-04-23T21:20:35.711 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:20:35.711 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2024-04-23T21:20:35.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2024-04-23T21:20:35.712 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2024-04-23T21:20:35.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2024-04-23T21:20:35.714 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2024-04-23T21:20:35.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:20:35.822+0000 7fe84d3e4640 -1 rbd::mirror::PoolReplayer: 0x55b55f3d1b00 init: failed to initialize remote pool poller: (2) No such file or directory 2024-04-23T21:20:36.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:20:36.062+0000 7fe84d3e4640 -1 rbd::mirror::PoolReplayer: 0x55b5604a8c00 init: failed to initialize remote pool poller: (2) No such file or directory 2024-04-23T21:20:37.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ CEPH_ARGS= 2024-04-23T21:20:37.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 pool init mirror_parent 2024-04-23T21:20:40.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:20:40.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ CEPH_ARGS= 2024-04-23T21:20:40.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 mirror pool enable --site-name cluster2 mirror pool 2024-04-23T21:20:40.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 mirror pool enable mirror_parent image 2024-04-23T21:20:40.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 namespace create mirror/ns1 2024-04-23T21:20:40.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 namespace create mirror/ns2 2024-04-23T21:20:40.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns1 pool 2024-04-23T21:20:40.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns2 image 2024-04-23T21:20:40.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -z ']' 2024-04-23T21:20:40.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -z ']' 2024-04-23T21:20:40.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ peer_add cluster2 mirror cluster1 2024-04-23T21:20:40.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:20:40.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:40.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:20:40.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:40.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local client_cluster=cluster1 2024-04-23T21:20:40.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:40.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local remote_cluster=cluster1 2024-04-23T21:20:40.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local uuid_var_name 2024-04-23T21:20:40.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:20:40.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local error_code 2024-04-23T21:20:40.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local peer_uuid 2024-04-23T21:20:40.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 16 32 2024-04-23T21:20:40.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:20:40.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 mirror pool peer add mirror cluster1 2024-04-23T21:20:40.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ peer_uuid=a9116b4f-737e-411f-986d-5eb4946f3748 2024-04-23T21:20:40.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=0 2024-04-23T21:20:40.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:20:40.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 -eq 17 ']' 2024-04-23T21:20:40.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 0 -eq 0 2024-04-23T21:20:40.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:20:40.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:20:40.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ peer_add cluster2 mirror_parent cluster1 2024-04-23T21:20:40.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:20:40.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:40.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror_parent 2024-04-23T21:20:40.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:40.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local client_cluster=cluster1 2024-04-23T21:20:40.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:40.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local remote_cluster=cluster1 2024-04-23T21:20:40.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local uuid_var_name 2024-04-23T21:20:40.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:20:40.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local error_code 2024-04-23T21:20:40.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local peer_uuid 2024-04-23T21:20:40.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 16 32 2024-04-23T21:20:40.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:20:40.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 mirror pool peer add mirror_parent cluster1 2024-04-23T21:20:41.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ peer_uuid=94463de6-2f93-4192-9619-85773d8fc322 2024-04-23T21:20:41.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=0 2024-04-23T21:20:41.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:20:41.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 -eq 17 ']' 2024-04-23T21:20:41.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 0 -eq 0 2024-04-23T21:20:41.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:20:41.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:20:41.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:20:41.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ start_mirrors cluster1 2024-04-23T21:20:41.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:20:41.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:20:41.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:41.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ start_mirror cluster1:0 2024-04-23T21:20:41.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:20:41.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:20:41.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:20:41.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:20:41.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:20:41.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:20:41.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:20:41.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:20:41.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:20:41.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:20:41.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:41.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:20:41.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:20:41.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:20:41.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:20:41.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:41.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ start_mirror cluster1:1 2024-04-23T21:20:41.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:20:41.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:20:41.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:20:41.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:20:41.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:20:41.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:20:41.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:20:41.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:20:41.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:20:41.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:20:41.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:41.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:20:41.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:20:41.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:20:41.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:20:41.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:41.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ start_mirror cluster1:2 2024-04-23T21:20:41.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:20:41.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:20:41.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:20:41.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:20:41.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:20:41.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:20:41.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:20:41.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:20:41.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:20:41.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:20:41.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:41.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:20:41.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:20:41.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:20:41.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:20:41.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:41.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ start_mirror cluster1:3 2024-04-23T21:20:41.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:20:41.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:20:41.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:20:41.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:20:41.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:20:41.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:20:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:20:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:20:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:20:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:20:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:20:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:20:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:20:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:20:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ start_mirrors cluster2 2024-04-23T21:20:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:20:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:20:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ start_mirror cluster2:0 2024-04-23T21:20:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2:0 2024-04-23T21:20:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:20:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster2:0 cluster instance 2024-04-23T21:20:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster2:0 2024-04-23T21:20:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:20:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:20:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster2 2024-04-23T21:20:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:20:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster2:0 ']' 2024-04-23T21:20:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster2 2024-04-23T21:20:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster2 2024-04-23T21:20:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:20:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:20:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:20:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:20:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ start_mirror cluster2:1 2024-04-23T21:20:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2:1 2024-04-23T21:20:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:20:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster2:1 cluster instance 2024-04-23T21:20:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster2:1 2024-04-23T21:20:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:20:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:20:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster2 2024-04-23T21:20:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:20:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster2:1 ']' 2024-04-23T21:20:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster2 2024-04-23T21:20:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster2 2024-04-23T21:20:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:20:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:20:41.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:20:41.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:20:41.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:41.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ start_mirror cluster2:2 2024-04-23T21:20:41.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2:2 2024-04-23T21:20:41.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:20:41.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster2:2 cluster instance 2024-04-23T21:20:41.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster2:2 2024-04-23T21:20:41.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:20:41.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:20:41.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster2 2024-04-23T21:20:41.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:20:41.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster2:2 ']' 2024-04-23T21:20:41.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster2 2024-04-23T21:20:41.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster2 2024-04-23T21:20:41.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:20:41.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:20:41.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:20:41.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:20:41.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:41.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ start_mirror cluster2:3 2024-04-23T21:20:41.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2:3 2024-04-23T21:20:41.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:20:41.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster2:3 cluster instance 2024-04-23T21:20:41.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster2:3 2024-04-23T21:20:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:20:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:20:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster2 2024-04-23T21:20:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:20:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster2:3 ']' 2024-04-23T21:20:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster2 2024-04-23T21:20:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster2 2024-04-23T21:20:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:20:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:20:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:20:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:20:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ testlog 'TEST: add image and test replay after client crashes' 2024-04-23T21:20:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ tee -a /tmp/tmp.8Tyc9aSFBO/rbd-mirror.test.log 2024-04-23T21:20:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ date '+%F %T' 2024-04-23T21:20:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo 2024-04-23 21:20:41 TEST: add image and test replay after client crashes 2024-04-23T21:20:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:2024-04-23 21:20:41 TEST: add image and test replay after client crashes 2024-04-23T21:20:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=test 2024-04-23T21:20:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror test journal 512M 2024-04-23T21:20:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:20:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:20:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:20:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:20:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:20:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror test 512M 2024-04-23T21:20:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:20:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:20:41.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:41.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:20:41.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:41.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:20:41.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 512M ']' 2024-04-23T21:20:41.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=512M 2024-04-23T21:20:41.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:20:41.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering,exclusive-lock,journaling mirror/test 2024-04-23T21:20:41.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:20:41.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:20:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:20:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:20:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:20:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror test 2024-04-23T21:20:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:20:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:20:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:20:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2024-04-23T21:20:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:20:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:20:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:20:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:20:41.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:20:41.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:20:41.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:20:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror test started 2024-04-23T21:20:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:20:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:20:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:20:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:20:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:20:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:20:42.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2024-04-23T21:20:42.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:20:42.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:20:42.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:20:42.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:20:42.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:20:42.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:20:42.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:20:42.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:20:42.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:20:42.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:20:42.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:42.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:20:42.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:20:42.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:20:42.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:20:42.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:20:42.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:20:42.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:20:42.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:20:42.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:42.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:20:42.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:20:42.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:20:42.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:42.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:20:42.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:20:42.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:20:42.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:20:42.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:20:42.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:20:42.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:20:42.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:20:42.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:20:42.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:20:42.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:42.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:20:42.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:20:42.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:20:42.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:20:42.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:20:42.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:20:42.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:20:42.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:20:42.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:20:42.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:20:42.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:20:42.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:20:42.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:20:42.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:20:42.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:42.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:20:42.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:20:42.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:20:42.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:20:42.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:20:42.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:20:42.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:20:42.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:20:42.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:42.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:20:42.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:20:42.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:20:42.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:42.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:20:42.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:20:42.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:20:42.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:20:42.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:20:42.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:20:42.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:20:42.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:20:42.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:20:42.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:20:42.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:42.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:20:42.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:20:42.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:20:42.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:20:42.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:20:42.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:20:42.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:20:42.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:20:42.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:20:42.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:20:42.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:20:42.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:20:42.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:20:42.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:20:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2024-04-23T21:20:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:20:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:20:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:20:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:20:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:20:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:20:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:20:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:20:42.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:20:42.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:20:42.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:42.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:20:42.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:20:42.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:20:42.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:20:42.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:20:42.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:20:42.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:20:42.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:20:42.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:20:42.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:20:42.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:42.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:20:42.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:20:42.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:20:42.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:20:42.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:20:42.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:20:42.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:20:42.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:20:42.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:20:42.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:20:42.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2024-04-23T21:20:42.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:20:42.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:20:42.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:20:42.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:42.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2024-04-23T21:20:42.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:20:42.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:20:42.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:20:42.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:20:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:20:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:20:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:20:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:20:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:20:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:20:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:20:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:20:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:20:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:20:42.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:20:42.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:20:42.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:20:42.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:20:42.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:20:42.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:20:42.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:42.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:20:42.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:20:42.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:20:42.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:20:42.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:20:42.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:20:42.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:20:42.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:20:42.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:20:42.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:20:42.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2024-04-23T21:20:42.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:20:42.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:20:42.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:20:42.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:20:42.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 1 2024-04-23T21:20:42.879 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:20:42.878+0000 7fe97bfc2640 -1 reset not still connected to 0x55a5aecb1520 2024-04-23T21:20:43.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:20:43.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:20:43.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:43.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:20:43.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:20:43.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:20:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:20:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:20:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:20:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:20:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:20:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:20:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:20:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:20:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:20:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:20:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:20:43.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:20:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:20:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:20:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:20:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:20:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:20:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:20:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:20:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:20:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:20:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:20:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:20:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:20:43.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:20:43.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:20:43.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:20:43.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:20:43.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:20:43.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:20:43.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:20:43.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:20:43.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:43.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:20:43.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:20:43.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:20:43.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:20:43.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:20:43.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:20:43.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:20:43.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:20:43.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:43.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:20:43.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:20:43.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:20:43.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:43.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:20:43.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:20:43.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:20:43.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:20:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:20:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:20:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:20:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:20:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:20:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:20:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:20:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:20:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:20:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:20:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:20:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:20:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:20:43.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:20:43.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:20:43.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:20:43.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:20:43.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:20:43.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:20:43.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:20:43.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:43.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2024-04-23T21:20:43.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:20:43.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:20:43.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:20:43.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:20:43.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:20:43.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:20:43.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:20:43.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:43.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:20:43.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:20:43.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:20:43.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:43.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:20:43.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:20:43.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:20:43.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:20:43.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:20:43.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:20:43.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:20:43.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:20:43.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:20:43.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:20:43.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:43.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:20:43.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:20:43.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:20:43.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:20:43.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:20:43.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:20:43.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:20:43.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:20:43.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:20:43.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:20:43.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2024-04-23T21:20:43.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:20:43.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:20:43.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:20:43.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:43.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2024-04-23T21:20:43.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:20:43.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:20:43.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:20:43.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:20:43.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:20:43.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:20:43.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:20:43.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:43.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:20:43.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:20:43.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:20:43.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:43.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:20:43.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:20:43.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:20:43.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:20:43.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:20:43.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:20:43.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:20:43.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:20:43.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:20:43.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:20:43.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:43.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:20:43.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:20:43.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:20:43.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:20:43.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:20:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:20:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:20:43.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:20:43.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:20:43.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:20:43.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2024-04-23T21:20:43.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:20:43.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:20:43.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:20:43.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:20:43.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 2 2024-04-23T21:20:45.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:20:45.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:20:45.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:45.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:20:45.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:20:45.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:20:45.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:20:45.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:20:45.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:20:45.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:20:45.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:20:45.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:45.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:20:45.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:20:45.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:20:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:20:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:20:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:20:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:20:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:20:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:20:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:20:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:20:45.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:20:45.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:20:45.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:45.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:20:45.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:20:45.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:20:45.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:20:45.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:20:45.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:20:45.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:20:45.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:20:45.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:20:45.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:20:45.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:20:45.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:20:45.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:20:45.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:20:45.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:45.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:20:45.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:20:45.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:20:45.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:20:45.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:20:45.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:20:45.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:20:45.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:20:45.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:45.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:20:45.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:20:45.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:20:45.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:45.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:20:45.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:20:45.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:20:45.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:20:45.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:20:45.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:20:45.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:20:45.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:20:45.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:20:45.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:20:45.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:45.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:20:45.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:20:45.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:20:45.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:20:45.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:20:45.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:20:45.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:20:45.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:20:45.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:20:45.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:20:45.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:20:46.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:20:46.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:20:46.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:20:46.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:46.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2024-04-23T21:20:46.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:20:46.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:20:46.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:20:46.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:20:46.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:20:46.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:20:46.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:20:46.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:46.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:20:46.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:20:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:20:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:20:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:20:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:20:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:20:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:20:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:20:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:20:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:20:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:20:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:20:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:20:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:20:46.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:20:46.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:20:46.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:20:46.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:20:46.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:20:46.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:20:46.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:20:46.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:20:46.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2024-04-23T21:20:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:20:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:20:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:20:46.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:46.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2024-04-23T21:20:46.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:20:46.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:20:46.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:20:46.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:20:46.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:20:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:20:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:20:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:20:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:20:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:20:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:20:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:20:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:20:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:20:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:20:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:20:46.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:20:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:20:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:20:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:20:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:20:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:20:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:20:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:20:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:20:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:20:46.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:20:46.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:20:46.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:20:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:20:46.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2024-04-23T21:20:46.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:20:46.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:20:46.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:20:46.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:20:46.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 4 2024-04-23T21:20:50.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:20:50.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:20:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:20:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:20:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:20:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:20:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:20:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:20:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:20:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:20:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:20:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:20:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:20:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:20:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:20:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:20:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:20:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:20:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:20:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:20:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:20:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:20:50.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:20:50.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:50.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:20:50.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:20:50.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:20:50.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:20:50.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:20:50.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:20:50.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:20:50.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:20:50.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:20:50.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:20:50.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:20:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:20:50.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:20:50.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:20:50.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:50.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:20:50.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:20:50.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:20:50.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:20:50.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:20:50.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:20:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:20:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:20:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:20:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:20:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:20:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:20:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:20:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:20:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:20:50.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:20:50.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:20:50.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:20:50.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:20:50.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:20:50.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:20:50.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:50.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:20:50.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:20:50.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:20:50.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:20:50.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:20:50.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:20:50.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:20:50.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:20:50.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:20:50.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:20:50.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:20:50.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:20:50.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:20:50.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:20:50.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:20:50.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2024-04-23T21:20:50.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:20:50.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:20:50.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:20:50.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:20:50.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:20:50.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:20:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:20:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:20:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:20:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:20:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:20:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:20:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:20:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:20:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:20:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:20:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:20:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:20:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:20:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:20:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:20:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:20:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:20:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:20:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:20:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:20:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:20:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:20:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:20:50.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:20:50.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:20:50.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:20:50.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2024-04-23T21:20:50.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:20:50.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:20:50.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:20:50.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:20:50.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:20:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:20:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:20:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:20:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ clean_snap_name= 2024-04-23T21:20:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 1 10 2024-04-23T21:20:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:20:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror test 2024-04-23T21:20:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:20:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:20:50.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:20:50.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:20:50.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=11 2024-04-23T21:20:50.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:20:50.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2024-04-23T21:20:51.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 59593728 bytes 2024-04-23T21:20:52.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 90429440 bytes 2024-04-23T21:20:54.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 136582144 bytes 2024-04-23T21:20:56.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 205828096 bytes 2024-04-23T21:20:58.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 245880832 bytes 2024-04-23T21:20:59.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:600 IOs, 289907712 bytes 2024-04-23T21:21:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:700 IOs, 338856960 bytes 2024-04-23T21:21:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:21:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:21:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:21:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2024-04-23T21:21:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:21:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:21:01.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:21:01.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:01.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:21:02.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2024-04-23T21:21:02.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:02.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:02.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:02.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:21:02.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:21:02.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:21:02.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:21:02.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:21:02.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2024-04-23T21:21:02.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='test: 2024-04-23T21:21:02.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: global_id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:21:02.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+replaying 2024-04-23T21:21:02.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: replaying, {"bytes_per_second":0.0,"entries_behind_primary":3,"entries_per_second":0.0,"non_primary_position":{},"primary_position":{"entry_tid":3,"object_number":3,"tag_tid":1}} 2024-04-23T21:21:02.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: service: mirror.2 on smithi150 2024-04-23T21:21:02.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:20:46 2024-04-23T21:21:02.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: peer_sites: 2024-04-23T21:21:02.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: name: cluster2 2024-04-23T21:21:02.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+stopped 2024-04-23T21:21:02.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: local image is primary 2024-04-23T21:21:02.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:20:44' 2024-04-23T21:21:02.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ state: .*up+replaying' 2024-04-23T21:21:02.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: state: up+replaying 2024-04-23T21:21:02.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ description: .*' 2024-04-23T21:21:02.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: description: replaying, {"bytes_per_second":0.0,"entries_behind_primary":3,"entries_per_second":0.0,"non_primary_position":{},"primary_position":{"entry_tid":3,"object_number":3,"tag_tid":1}} 2024-04-23T21:21:02.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:21:02.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:21:02.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^up+' 2024-04-23T21:21:02.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:21:02.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep 'service: *mirror..* on ' 2024-04-23T21:21:02.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: service: mirror.2 on smithi150 2024-04-23T21:21:02.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update 2024-04-23T21:21:02.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-04-23T21:21:02.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:20:46' 2024-04-23T21:21:02.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2024-04-23 21:20:46' 2024-04-23T21:21:02.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:02.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:02.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:02.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:21:02.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'prev_last_update=2024-04-23 21:20:46' 2024-04-23T21:21:02.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:21:02.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:21:02.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-04-23T21:21:02.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='ERRORERROROK142174271mirror.2smithi15019.0.0-3212-g8254cb5atrueOK42184274mirror.1smithi15019.0.0-3212-g8254cb5afalseOK42224277mirror.3smithi15019.0.0-3212-g8254cb5afalseOK4262mirror.0smithi15019.0.0-3212-g8254cb5afalseERRORunable to initialize remote pool pollerteste56bd005-f8a9-476d-a8fc-713b4e8cd286up+replayingreplaying, {"bytes_per_second":0.0,"entries_behind_primary":3,"entries_per_second":0.0,"non_primary_position":{},"primary_position":{"entry_tid":3,"object_number":3,"tag_tid":1}}42174271mirror.2smithi1502024-04-23 21:20:46cluster267253cce-c621-4f62-a184-da88591999d7up+stoppedlocal image is primary2024-04-23 21:20:44' 2024-04-23T21:21:02.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update state 2024-04-23T21:21:02.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2024-04-23T21:21:02.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:20:46' 2024-04-23T21:21:02.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2024-04-23T21:21:02.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ state=up+replaying 2024-04-23T21:21:02.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:21:02.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep up+replaying 2024-04-23T21:21:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:21:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap1 2024-04-23T21:21:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap1 2024-04-23T21:21:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:21:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap1 2024-04-23T21:21:02.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap1 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:21:05.709 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2024-04-23T21:21:05.709 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2024-04-23T21:21:05.710 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2024-04-23T21:21:05.710 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:21:05.711+0000 7ff3a5ad7640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 35452) UID: 1000 2024-04-23T21:21:06.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:21:06.070+0000 7fe84d3e4640 -1 rbd::mirror::Mirror: 0x55b560132000 update_pool_replayers: restarting failed pool replayer for uuid: 38aea5d5-3d8e-44a1-bf69-12083687b763 cluster: cluster2 client: client.mirror 2024-04-23T21:21:06.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:21:06.115+0000 7fe84d3e4640 -1 rbd::mirror::Mirror: 0x55b560132000 update_pool_replayers: restarting failed pool replayer for uuid: e8cfaf48-e208-4de2-a791-8072a511293d cluster: cluster2 client: client.mirror 2024-04-23T21:21:10.710 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2024-04-23T21:21:10.710 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:21:10.711 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2024-04-23T21:21:10.711 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2024-04-23T21:21:10.711 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2024-04-23T21:21:10.714 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2024-04-23T21:21:10.714 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2024-04-23T21:21:32.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror test 2024-04-23T21:21:32.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:32.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:32.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:32.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2024-04-23T21:21:32.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:32.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:32.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:32.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:21:32.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:21:32.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:32.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:21:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror test started 2024-04-23T21:21:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:21:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:21:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:21:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2024-04-23T21:21:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:21:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:21:33.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:21:33.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:33.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:21:33.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:21:33.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:33.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:21:33.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:21:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:21:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:21:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:21:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:21:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:21:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:21:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:21:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:21:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:21:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:21:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:21:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:21:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:21:33.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:33.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:33.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:21:33.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:21:33.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:21:33.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:21:33.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:33.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:33.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:33.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:33.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:21:33.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:33.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:21:33.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:33.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:21:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:21:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:21:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:21:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:21:33.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:21:33.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:33.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:33.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:33.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:21:33.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:21:33.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:21:33.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:33.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:21:33.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:21:33.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:21:33.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:21:33.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:33.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:33.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:21:33.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:21:33.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:21:33.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:21:33.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:33.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:33.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:33.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:33.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:21:33.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:33.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:21:33.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2024-04-23T21:21:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:21:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:21:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:21:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:33.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:21:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:21:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:21:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:21:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:21:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:21:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:21:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:21:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:21:33.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:33.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:33.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:21:33.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:21:33.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:21:33.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:21:33.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:33.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:33.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:33.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:33.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2024-04-23T21:21:33.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:21:33.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:21:33.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:21:33.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:21:33.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:21:33.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:21:33.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:21:33.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:33.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:21:33.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:21:33.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:21:33.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:33.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:33.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:21:33.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:21:33.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:21:33.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:21:33.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:33.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:33.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:21:33.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:21:33.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:33.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:33.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:34.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:34.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:34.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:34.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:34.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:34.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:34.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:34.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:34.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:34.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:34.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:34.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:34.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:34.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:34.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:34.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:34.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:34.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:34.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:34.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:34.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:34.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:34.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:34.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:34.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:34.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:34.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:34.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:34.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:34.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:34.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:34.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:34.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:34.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:34.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:34.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:34.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:34.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:34.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:34.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:34.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:34.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:34.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:34.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:34.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:34.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:34.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:34.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:34.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:34.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:34.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:34.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:34.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:34.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:34.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:34.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:34.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:34.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:34.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:34.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:34.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:34.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:34.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:34.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:34.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:34.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:34.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:34.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:34.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:34.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:34.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:34.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:34.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:34.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:34.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:34.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:34.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:34.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:34.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:34.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:34.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:34.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:34.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:34.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:34.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:34.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:34.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:34.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:34.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=18, tag_tid=2, entry_tid=16886], [object_number=17, tag_tid=2, entry_tid=16885], [object_number=16, tag_tid=2, entry_tid=16884], [object_number=19, tag_tid=2, entry_tid=16883]]], state=connected] 2024-04-23T21:21:34.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:34.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:34.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:34.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:34.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:34.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:34.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:34.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:34.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:34.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:34.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:34.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:34.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:34.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:34.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:34.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:34.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:34.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:34.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:34.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:34.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:34.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=18, tag_tid=2, entry_tid=16886], [object_number=17, tag_tid=2, entry_tid=16885], [object_number=16, tag_tid=2, entry_tid=16884], [object_number=19, tag_tid=2, entry_tid=16883]]], state=connected] 2024-04-23T21:21:34.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:34.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=18, tag_tid=2, entry_tid=16886' 2024-04-23T21:21:34.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=18, tag_tid=2, entry_tid=16886' 2024-04-23T21:21:34.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=18, tag_tid=2, entry_tid=16886' '!=' '' 2024-04-23T21:21:34.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:34.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=18, tag_tid=2, entry_tid=16886' = '' 2024-04-23T21:21:34.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=18, tag_tid=2, entry_tid=16886' 2024-04-23T21:21:34.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:34.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:34.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:34.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:34.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=18, tag_tid=2, entry_tid=16886' 2024-04-23T21:21:34.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:34.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:34.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:34.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:34.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:34.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:34.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:34.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=18, tag_tid=2, entry_tid=16886' 2024-04-23T21:21:34.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:34.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=16886 2024-04-23T21:21:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 16886 2024-04-23T21:21:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:34.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:34.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:34.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:34.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:34.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:34.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:34.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:34.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:34.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:34.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:34.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:34.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:34.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:34.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:34.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:34.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:34.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:34.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:34.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:34.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:34.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:34.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:34.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:34.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:34.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:34.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:34.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:34.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:34.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:34.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:34.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:34.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:34.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:34.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:34.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:34.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:34.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:34.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:34.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:34.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:34.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:34.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:34.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:34.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:34.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:34.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:34.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:34.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:34.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:34.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:34.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:34.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:34.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:34.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:34.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:34.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:34.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:34.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:34.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:34.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:34.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:34.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:34.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:34.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:34.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:34.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:34.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:34.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:34.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:35.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:35.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:35.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:35.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:35.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:35.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:35.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:35.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:35.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:35.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:35.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:35.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:35.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:35.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:35.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:35.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:35.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:35.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:35.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:35.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:35.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=19, tag_tid=2, entry_tid=17171], [object_number=18, tag_tid=2, entry_tid=17170], [object_number=17, tag_tid=2, entry_tid=17169], [object_number=16, tag_tid=2, entry_tid=17168]]], state=connected] 2024-04-23T21:21:35.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:35.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:35.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:35.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:35.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:35.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:35.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:35.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:35.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:35.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:35.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:35.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:35.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:35.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:35.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:35.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:35.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:35.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:35.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:35.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:35.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:35.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=19, tag_tid=2, entry_tid=17171], [object_number=18, tag_tid=2, entry_tid=17170], [object_number=17, tag_tid=2, entry_tid=17169], [object_number=16, tag_tid=2, entry_tid=17168]]], state=connected] 2024-04-23T21:21:35.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:35.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=19, tag_tid=2, entry_tid=17171' 2024-04-23T21:21:35.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=19, tag_tid=2, entry_tid=17171' 2024-04-23T21:21:35.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=19, tag_tid=2, entry_tid=17171' '!=' 'object_number=18, tag_tid=2, entry_tid=16886' 2024-04-23T21:21:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=19, tag_tid=2, entry_tid=17171' = 'object_number=18, tag_tid=2, entry_tid=16886' 2024-04-23T21:21:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=19, tag_tid=2, entry_tid=17171' 2024-04-23T21:21:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:35.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:35.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=19, tag_tid=2, entry_tid=17171' 2024-04-23T21:21:35.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:35.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:35.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:35.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:35.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:35.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:35.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=19, tag_tid=2, entry_tid=17171' 2024-04-23T21:21:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:35.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=17171 2024-04-23T21:21:35.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 17171 2024-04-23T21:21:35.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:35.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:35.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:35.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:35.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:35.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:35.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:35.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:35.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:35.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:35.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:35.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:35.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:35.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:35.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:35.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:35.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:35.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:35.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:35.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:35.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:35.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:35.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:35.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:35.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:35.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:35.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:35.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:35.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:35.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:35.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:35.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:35.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:35.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:35.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:35.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:35.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:35.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:35.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:35.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:35.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:35.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:35.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:35.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:35.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:35.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:35.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:35.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:35.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:35.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:35.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:35.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:35.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:35.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:35.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:35.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:35.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:35.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:35.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:35.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:35.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:35.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:35.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:35.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:35.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:35.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:35.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:35.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:35.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:35.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:35.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:35.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:35.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:35.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:35.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:35.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:35.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:35.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:35.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:35.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:35.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:35.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:35.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:35.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:35.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:35.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:35.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:35.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:35.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:35.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:35.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:35.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:35.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:35.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:35.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:35.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:35.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:35.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:35.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:35.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:35.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:35.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:35.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:35.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=17, tag_tid=2, entry_tid=17457], [object_number=16, tag_tid=2, entry_tid=17456], [object_number=19, tag_tid=2, entry_tid=17455], [object_number=18, tag_tid=2, entry_tid=17454]]], state=connected] 2024-04-23T21:21:35.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:35.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:35.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:35.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:35.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:35.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:35.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:35.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:35.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:35.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:35.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:35.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:35.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:35.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:35.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:35.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:35.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:35.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:35.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:35.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:35.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:35.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=17, tag_tid=2, entry_tid=17457], [object_number=16, tag_tid=2, entry_tid=17456], [object_number=19, tag_tid=2, entry_tid=17455], [object_number=18, tag_tid=2, entry_tid=17454]]], state=connected] 2024-04-23T21:21:35.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:35.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=17, tag_tid=2, entry_tid=17457' 2024-04-23T21:21:35.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=17, tag_tid=2, entry_tid=17457' 2024-04-23T21:21:35.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=17, tag_tid=2, entry_tid=17457' '!=' 'object_number=19, tag_tid=2, entry_tid=17171' 2024-04-23T21:21:35.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:35.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=17, tag_tid=2, entry_tid=17457' = 'object_number=19, tag_tid=2, entry_tid=17171' 2024-04-23T21:21:35.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=17, tag_tid=2, entry_tid=17457' 2024-04-23T21:21:35.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:35.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:35.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:35.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:35.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=17, tag_tid=2, entry_tid=17457' 2024-04-23T21:21:35.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:35.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:35.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:35.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:35.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:35.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:35.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:35.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=17, tag_tid=2, entry_tid=17457' 2024-04-23T21:21:35.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:35.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:35.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=17457 2024-04-23T21:21:35.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 17457 2024-04-23T21:21:35.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:35.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:35.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:36.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:36.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:36.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:36.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:36.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:36.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:36.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:36.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:36.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:36.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:36.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:36.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:36.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:36.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:36.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:36.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:36.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:36.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:36.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:36.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:36.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:36.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:36.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:36.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:36.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:36.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:36.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:36.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:36.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:36.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:36.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:36.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:36.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:36.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:36.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:36.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:36.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:36.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:36.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:36.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:36.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:36.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:36.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:36.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:36.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:36.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:36.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:36.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:36.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:36.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:36.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:36.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:36.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:36.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:36.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:36.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:36.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:36.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:36.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:36.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:36.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:36.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:36.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:36.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:36.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:36.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:36.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:36.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:36.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:36.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:36.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:36.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:36.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:36.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:36.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:36.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:36.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:36.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:36.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:36.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:36.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:36.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:36.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:36.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:36.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:36.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:36.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:36.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:36.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:36.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:36.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:36.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:36.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:36.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:36.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:36.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:36.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:36.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:36.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:36.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:36.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:36.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:36.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:36.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:36.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:36.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:36.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:36.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:36.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:36.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:36.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:36.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:36.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:36.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:36.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:36.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:36.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:36.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:36.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:36.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:36.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:36.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:36.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:36.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:36.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:36.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:36.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:36.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:36.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:36.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:36.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:36.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:36.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:36.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:36.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:36.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:36.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:36.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:36.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:36.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:36.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:36.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:36.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:36.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:36.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:36.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:36.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:36.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:36.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:36.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:36.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=18, tag_tid=2, entry_tid=17770], [object_number=17, tag_tid=2, entry_tid=17769], [object_number=16, tag_tid=2, entry_tid=17768], [object_number=19, tag_tid=2, entry_tid=17767]]], state=connected] 2024-04-23T21:21:36.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:36.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:36.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:36.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:36.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:36.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:36.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:36.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:36.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:36.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:36.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:36.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=18, tag_tid=2, entry_tid=17770], [object_number=17, tag_tid=2, entry_tid=17769], [object_number=16, tag_tid=2, entry_tid=17768], [object_number=19, tag_tid=2, entry_tid=17767]]], state=connected] 2024-04-23T21:21:36.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:36.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=18, tag_tid=2, entry_tid=17770' 2024-04-23T21:21:36.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=18, tag_tid=2, entry_tid=17770' 2024-04-23T21:21:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=18, tag_tid=2, entry_tid=17770' '!=' 'object_number=17, tag_tid=2, entry_tid=17457' 2024-04-23T21:21:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=18, tag_tid=2, entry_tid=17770' = 'object_number=17, tag_tid=2, entry_tid=17457' 2024-04-23T21:21:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=18, tag_tid=2, entry_tid=17770' 2024-04-23T21:21:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=18, tag_tid=2, entry_tid=17770' 2024-04-23T21:21:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:36.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:36.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:36.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:36.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:36.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:36.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=18, tag_tid=2, entry_tid=17770' 2024-04-23T21:21:36.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:36.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:36.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=17770 2024-04-23T21:21:36.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 17770 2024-04-23T21:21:36.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:36.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:36.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:36.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:36.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:36.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:36.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:36.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:36.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:36.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:36.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:36.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:36.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:36.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:36.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:36.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:36.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:36.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:36.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:36.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:36.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:36.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:36.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:36.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:36.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:36.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:36.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:36.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:36.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:36.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:36.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:36.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:36.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:36.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:36.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:36.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:36.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:36.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:36.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:36.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:36.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:36.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:36.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:36.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:36.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:36.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:36.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:36.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:36.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:36.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:36.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:36.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:36.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:36.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:36.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:36.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:36.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:36.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:36.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:36.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:36.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:36.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:36.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:36.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:36.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:36.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:36.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:36.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:36.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:36.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:36.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:36.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:36.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:36.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:36.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:36.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:36.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:36.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:36.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:36.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:36.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:36.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:36.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:36.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:36.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:36.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:36.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:36.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:36.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:36.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:36.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:36.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:36.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:36.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:36.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:36.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:36.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:36.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:36.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:36.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:36.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:36.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:36.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:36.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:36.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:36.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:36.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:36.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=19, tag_tid=2, entry_tid=18119], [object_number=18, tag_tid=2, entry_tid=18118], [object_number=17, tag_tid=2, entry_tid=18117], [object_number=16, tag_tid=2, entry_tid=18116]]], state=connected] 2024-04-23T21:21:36.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:36.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:36.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:36.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:36.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:36.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:36.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:36.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:36.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:36.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:36.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:36.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:36.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:36.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:36.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:36.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:36.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:37.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:37.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:37.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:37.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:37.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=19, tag_tid=2, entry_tid=18119], [object_number=18, tag_tid=2, entry_tid=18118], [object_number=17, tag_tid=2, entry_tid=18117], [object_number=16, tag_tid=2, entry_tid=18116]]], state=connected] 2024-04-23T21:21:37.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:37.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=19, tag_tid=2, entry_tid=18119' 2024-04-23T21:21:37.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=19, tag_tid=2, entry_tid=18119' 2024-04-23T21:21:37.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=19, tag_tid=2, entry_tid=18119' '!=' 'object_number=18, tag_tid=2, entry_tid=17770' 2024-04-23T21:21:37.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:37.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=19, tag_tid=2, entry_tid=18119' = 'object_number=18, tag_tid=2, entry_tid=17770' 2024-04-23T21:21:37.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=19, tag_tid=2, entry_tid=18119' 2024-04-23T21:21:37.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:37.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:37.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:37.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:37.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=19, tag_tid=2, entry_tid=18119' 2024-04-23T21:21:37.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:37.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:37.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:37.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:37.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:37.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:37.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:37.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=19, tag_tid=2, entry_tid=18119' 2024-04-23T21:21:37.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:37.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:37.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=18119 2024-04-23T21:21:37.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 18119 2024-04-23T21:21:37.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:37.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:37.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:37.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:37.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:37.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:37.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:37.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:37.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:37.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:37.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:37.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:37.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:37.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:37.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:37.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:37.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:37.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:37.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:37.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:37.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:37.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:37.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:37.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:37.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:37.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:37.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:37.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:37.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:37.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:37.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:37.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:37.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:37.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:37.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:37.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:37.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:37.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:37.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:37.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:37.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:37.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:37.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:37.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:37.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:37.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:37.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:37.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:37.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:37.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:37.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:37.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:37.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:37.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:37.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:37.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:37.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:37.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:37.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:37.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:37.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:37.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:37.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:37.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:37.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:37.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:37.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:37.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:37.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:37.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:37.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:37.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:37.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:37.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:37.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:37.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:37.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:37.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:37.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:37.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:37.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:37.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:37.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:37.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:37.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:37.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:37.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:37.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:37.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:37.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:37.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:37.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:37.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:37.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:37.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:37.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:37.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:37.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:37.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:37.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:37.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:37.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:37.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:37.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:37.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:37.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:37.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:37.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:37.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:37.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:37.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:37.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:37.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:37.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:37.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:37.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:37.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:37.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:37.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:37.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:37.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:37.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:37.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:37.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:37.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:37.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:37.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:37.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:37.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:37.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:37.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:37.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:37.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:37.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:37.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:37.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:37.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:37.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:37.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:37.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:37.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:37.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:37.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:37.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:37.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:37.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:37.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:37.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:37.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:37.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:37.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:37.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:37.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:37.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:37.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=16, tag_tid=2, entry_tid=18416], [object_number=19, tag_tid=2, entry_tid=18415], [object_number=18, tag_tid=2, entry_tid=18414], [object_number=17, tag_tid=2, entry_tid=18413]]], state=connected] 2024-04-23T21:21:37.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:37.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:37.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:37.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:37.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:37.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:37.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:37.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:37.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:37.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:37.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:37.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:37.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:37.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:37.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:37.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:37.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:37.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:37.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:37.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:37.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:37.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=16, tag_tid=2, entry_tid=18416], [object_number=19, tag_tid=2, entry_tid=18415], [object_number=18, tag_tid=2, entry_tid=18414], [object_number=17, tag_tid=2, entry_tid=18413]]], state=connected] 2024-04-23T21:21:37.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:37.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=16, tag_tid=2, entry_tid=18416' 2024-04-23T21:21:37.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=16, tag_tid=2, entry_tid=18416' 2024-04-23T21:21:37.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=16, tag_tid=2, entry_tid=18416' '!=' 'object_number=19, tag_tid=2, entry_tid=18119' 2024-04-23T21:21:37.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:37.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=16, tag_tid=2, entry_tid=18416' = 'object_number=19, tag_tid=2, entry_tid=18119' 2024-04-23T21:21:37.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=16, tag_tid=2, entry_tid=18416' 2024-04-23T21:21:37.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:37.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:37.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:37.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:37.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=16, tag_tid=2, entry_tid=18416' 2024-04-23T21:21:37.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:37.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:37.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:37.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:37.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:37.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:37.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:37.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=16, tag_tid=2, entry_tid=18416' 2024-04-23T21:21:37.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:37.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:37.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=18416 2024-04-23T21:21:37.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 18416 2024-04-23T21:21:37.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:37.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:37.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:37.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:37.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:37.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:37.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:37.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:37.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:37.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:37.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:37.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:37.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:37.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:37.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:37.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:37.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:37.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:37.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:37.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:37.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:37.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:37.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:37.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:37.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:37.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:37.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:37.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:37.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:37.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:37.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:37.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:37.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:37.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:37.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:37.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:37.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:37.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:37.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:37.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:37.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:37.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:37.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:37.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:37.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:37.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:37.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:37.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:37.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:37.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:37.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:37.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:37.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:37.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:37.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:37.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:37.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:37.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:37.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:37.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:37.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:37.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:37.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:37.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:37.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:37.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:37.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:37.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:37.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:37.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:37.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:37.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:37.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:37.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:37.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:37.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:37.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:37.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:37.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:37.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:37.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:38.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:38.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:38.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:38.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:38.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:38.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:38.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:38.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:38.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:38.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:38.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:38.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:38.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:38.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:38.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:38.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:38.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:38.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:38.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:38.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:38.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:38.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:38.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:38.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:38.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:38.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:38.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:38.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:38.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:38.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:38.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:38.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:38.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:38.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:38.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:38.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:38.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:38.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:38.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:38.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:38.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:38.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:38.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:38.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:38.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:38.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:38.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:38.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:38.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:38.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=18, tag_tid=2, entry_tid=18742], [object_number=17, tag_tid=2, entry_tid=18741], [object_number=16, tag_tid=2, entry_tid=18740], [object_number=19, tag_tid=2, entry_tid=18739]]], state=connected] 2024-04-23T21:21:38.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:38.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:38.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:38.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:38.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:38.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:38.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:38.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:38.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:38.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:38.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:38.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:38.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:38.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:38.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:38.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:38.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:38.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:38.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:38.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:38.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:38.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=18, tag_tid=2, entry_tid=18742], [object_number=17, tag_tid=2, entry_tid=18741], [object_number=16, tag_tid=2, entry_tid=18740], [object_number=19, tag_tid=2, entry_tid=18739]]], state=connected] 2024-04-23T21:21:38.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:38.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=18, tag_tid=2, entry_tid=18742' 2024-04-23T21:21:38.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=18, tag_tid=2, entry_tid=18742' 2024-04-23T21:21:38.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=18, tag_tid=2, entry_tid=18742' '!=' 'object_number=16, tag_tid=2, entry_tid=18416' 2024-04-23T21:21:38.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:38.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=18, tag_tid=2, entry_tid=18742' = 'object_number=16, tag_tid=2, entry_tid=18416' 2024-04-23T21:21:38.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=18, tag_tid=2, entry_tid=18742' 2024-04-23T21:21:38.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:38.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:38.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:38.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:38.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=18, tag_tid=2, entry_tid=18742' 2024-04-23T21:21:38.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:38.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:38.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:38.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:38.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:38.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:38.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:38.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=18, tag_tid=2, entry_tid=18742' 2024-04-23T21:21:38.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:38.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:38.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=18742 2024-04-23T21:21:38.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 18742 2024-04-23T21:21:38.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:38.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:38.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:38.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:38.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:38.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:38.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:38.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:38.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:38.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:38.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:38.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:38.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:38.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:38.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:38.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:38.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:38.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:38.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:38.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:38.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:38.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:38.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:38.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:38.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:38.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:38.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:38.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:38.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:38.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:38.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:38.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:38.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:38.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:38.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:38.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:38.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:38.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:38.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:38.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:38.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:38.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:38.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:38.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:38.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:38.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:38.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:38.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:38.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:38.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:38.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:38.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:38.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:38.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:38.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:38.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:38.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:38.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:38.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:38.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:38.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:38.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:38.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:38.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:38.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:38.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:38.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:38.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:38.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:38.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:38.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:38.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:38.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:38.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:38.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:38.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:38.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:38.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:38.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:38.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:38.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:38.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:38.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:38.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:38.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:38.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:38.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:38.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:38.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:38.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:38.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:38.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:38.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:38.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:38.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:38.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:38.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:38.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:38.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:38.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:38.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:38.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:38.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:38.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:38.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:38.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:38.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:38.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:38.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:38.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:38.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:38.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:38.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:38.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:38.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:38.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:38.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:38.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=16, tag_tid=2, entry_tid=19076], [object_number=19, tag_tid=2, entry_tid=19075], [object_number=18, tag_tid=2, entry_tid=19074], [object_number=17, tag_tid=2, entry_tid=19073]]], state=connected] 2024-04-23T21:21:38.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:38.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:38.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:38.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:38.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:38.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:38.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:38.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:38.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:38.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:38.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:38.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:38.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:38.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:38.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:38.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:38.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:38.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:38.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:38.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:38.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:38.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=16, tag_tid=2, entry_tid=19076], [object_number=19, tag_tid=2, entry_tid=19075], [object_number=18, tag_tid=2, entry_tid=19074], [object_number=17, tag_tid=2, entry_tid=19073]]], state=connected] 2024-04-23T21:21:38.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:38.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=16, tag_tid=2, entry_tid=19076' 2024-04-23T21:21:38.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=16, tag_tid=2, entry_tid=19076' 2024-04-23T21:21:38.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=16, tag_tid=2, entry_tid=19076' '!=' 'object_number=18, tag_tid=2, entry_tid=18742' 2024-04-23T21:21:38.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:38.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=16, tag_tid=2, entry_tid=19076' = 'object_number=18, tag_tid=2, entry_tid=18742' 2024-04-23T21:21:38.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=16, tag_tid=2, entry_tid=19076' 2024-04-23T21:21:38.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:38.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:38.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:38.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:38.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=16, tag_tid=2, entry_tid=19076' 2024-04-23T21:21:38.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:38.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:38.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:38.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:38.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:38.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:38.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:38.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=16, tag_tid=2, entry_tid=19076' 2024-04-23T21:21:38.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:38.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:38.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=19076 2024-04-23T21:21:38.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 19076 2024-04-23T21:21:38.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:38.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:38.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:39.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:39.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:39.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:39.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:39.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:39.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:39.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:39.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:39.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:39.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:39.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:39.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:39.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:39.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:39.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:39.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:39.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:39.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:39.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:39.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:39.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:39.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:39.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:39.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:39.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:39.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:39.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:39.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:39.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:39.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:39.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:39.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:39.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:39.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:39.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:39.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:39.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:39.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:39.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:39.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:39.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:39.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:39.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:39.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:39.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:39.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:39.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:39.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:39.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:39.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:39.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:39.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:39.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:39.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:39.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:39.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:39.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:39.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:39.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:39.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:39.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:39.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:39.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:39.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:39.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:39.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:39.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:39.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:39.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:39.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:39.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:39.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:39.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:39.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:39.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:39.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:39.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:39.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:39.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:39.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:39.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:39.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:39.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:39.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:39.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:39.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:39.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:39.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:39.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:39.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:39.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:39.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:39.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:39.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:39.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:39.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:39.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:39.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=18, tag_tid=2, entry_tid=19406], [object_number=17, tag_tid=2, entry_tid=19405], [object_number=16, tag_tid=2, entry_tid=19404], [object_number=19, tag_tid=2, entry_tid=19403]]], state=connected] 2024-04-23T21:21:39.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:39.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:39.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:39.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:39.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:39.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:39.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:39.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:39.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:39.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:39.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:39.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:39.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:39.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:39.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:39.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:39.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:39.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:39.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:39.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:39.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:39.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=18, tag_tid=2, entry_tid=19406], [object_number=17, tag_tid=2, entry_tid=19405], [object_number=16, tag_tid=2, entry_tid=19404], [object_number=19, tag_tid=2, entry_tid=19403]]], state=connected] 2024-04-23T21:21:39.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:39.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=18, tag_tid=2, entry_tid=19406' 2024-04-23T21:21:39.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=18, tag_tid=2, entry_tid=19406' 2024-04-23T21:21:39.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=18, tag_tid=2, entry_tid=19406' '!=' 'object_number=16, tag_tid=2, entry_tid=19076' 2024-04-23T21:21:39.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:39.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=18, tag_tid=2, entry_tid=19406' = 'object_number=16, tag_tid=2, entry_tid=19076' 2024-04-23T21:21:39.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=18, tag_tid=2, entry_tid=19406' 2024-04-23T21:21:39.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:39.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:39.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:39.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:39.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=18, tag_tid=2, entry_tid=19406' 2024-04-23T21:21:39.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:39.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:39.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:39.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:39.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:39.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:39.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:39.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=18, tag_tid=2, entry_tid=19406' 2024-04-23T21:21:39.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:39.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:39.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=19406 2024-04-23T21:21:39.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 19406 2024-04-23T21:21:39.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:39.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:39.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:39.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:39.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:39.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:39.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:39.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:39.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:39.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:39.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:39.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:39.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:39.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:39.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:39.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:39.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:39.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:39.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:39.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:39.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:39.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:39.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:39.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:39.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:39.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:39.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:39.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:39.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:39.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:39.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:39.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:39.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:39.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:39.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:39.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:39.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:39.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:39.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:39.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:39.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:39.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:39.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:39.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:39.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:39.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:39.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:39.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:39.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:39.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:39.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:39.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:39.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:39.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:39.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:39.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:39.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:39.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:39.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:39.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:39.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:39.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:39.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:39.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:39.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:39.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:39.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:39.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:39.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:39.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:39.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:39.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:39.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:39.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:39.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:39.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:39.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:39.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:39.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:39.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:39.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:39.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:39.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:39.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:39.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:39.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:39.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:39.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:39.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:39.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:39.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:39.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:39.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:39.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:39.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:39.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:39.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:39.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:39.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:39.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:39.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:39.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:39.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:39.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:39.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:39.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:39.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:39.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:39.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:39.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:39.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:40.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:40.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=18, tag_tid=2, entry_tid=19630], [object_number=17, tag_tid=2, entry_tid=19629], [object_number=16, tag_tid=2, entry_tid=19628], [object_number=19, tag_tid=2, entry_tid=19627]]], state=connected] 2024-04-23T21:21:40.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:40.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:40.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:40.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:40.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:40.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:40.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:40.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:40.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:40.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:40.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:40.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:40.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:40.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:40.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:40.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:40.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:40.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:40.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:40.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:40.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:40.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=18, tag_tid=2, entry_tid=19630], [object_number=17, tag_tid=2, entry_tid=19629], [object_number=16, tag_tid=2, entry_tid=19628], [object_number=19, tag_tid=2, entry_tid=19627]]], state=connected] 2024-04-23T21:21:40.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:40.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=18, tag_tid=2, entry_tid=19630' 2024-04-23T21:21:40.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=18, tag_tid=2, entry_tid=19630' 2024-04-23T21:21:40.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=18, tag_tid=2, entry_tid=19630' '!=' 'object_number=18, tag_tid=2, entry_tid=19406' 2024-04-23T21:21:40.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:40.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=18, tag_tid=2, entry_tid=19630' = 'object_number=18, tag_tid=2, entry_tid=19406' 2024-04-23T21:21:40.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=18, tag_tid=2, entry_tid=19630' 2024-04-23T21:21:40.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:40.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:40.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:40.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:40.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=18, tag_tid=2, entry_tid=19630' 2024-04-23T21:21:40.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:40.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:40.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:40.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:40.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=18, tag_tid=2, entry_tid=19630' 2024-04-23T21:21:40.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:40.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:40.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=19630 2024-04-23T21:21:40.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 19630 2024-04-23T21:21:40.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:40.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:40.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:40.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:40.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:40.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:40.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:40.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:40.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:40.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:40.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:40.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:40.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:40.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:40.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:40.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:40.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:40.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:40.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:40.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:40.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:40.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:40.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:40.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:40.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:40.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:40.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:40.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:40.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:40.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:40.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:40.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:40.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:40.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:40.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:40.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:40.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:40.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:40.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:40.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:40.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:40.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:40.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:40.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:40.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:40.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:40.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:40.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:40.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:40.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:40.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:40.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:40.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:40.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:40.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:40.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:40.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:40.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:40.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:40.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:40.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:40.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:40.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:40.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:40.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:40.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:40.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:40.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:40.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:40.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:40.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:40.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:40.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:40.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:40.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:40.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:40.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:40.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:40.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:40.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:40.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:40.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:40.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:40.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:40.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:40.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:40.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:40.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:40.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:40.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:40.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:40.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:40.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:40.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:40.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:40.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:40.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:40.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:40.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:40.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:40.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:40.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:40.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:40.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:40.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:40.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:40.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=16, tag_tid=2, entry_tid=19948], [object_number=19, tag_tid=2, entry_tid=19947], [object_number=18, tag_tid=2, entry_tid=19946], [object_number=17, tag_tid=2, entry_tid=19945]]], state=connected] 2024-04-23T21:21:40.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:40.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:40.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:40.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:40.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:40.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:40.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:40.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:40.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:40.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:40.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:40.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:40.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:40.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:40.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:40.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=16, tag_tid=2, entry_tid=19948], [object_number=19, tag_tid=2, entry_tid=19947], [object_number=18, tag_tid=2, entry_tid=19946], [object_number=17, tag_tid=2, entry_tid=19945]]], state=connected] 2024-04-23T21:21:40.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:40.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=16, tag_tid=2, entry_tid=19948' 2024-04-23T21:21:40.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=16, tag_tid=2, entry_tid=19948' 2024-04-23T21:21:40.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=16, tag_tid=2, entry_tid=19948' '!=' 'object_number=18, tag_tid=2, entry_tid=19630' 2024-04-23T21:21:40.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:40.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=16, tag_tid=2, entry_tid=19948' = 'object_number=18, tag_tid=2, entry_tid=19630' 2024-04-23T21:21:40.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=16, tag_tid=2, entry_tid=19948' 2024-04-23T21:21:40.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:40.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:40.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:40.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:40.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=16, tag_tid=2, entry_tid=19948' 2024-04-23T21:21:40.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:40.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:40.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:40.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:40.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:40.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:40.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:40.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=16, tag_tid=2, entry_tid=19948' 2024-04-23T21:21:40.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:40.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:40.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=19948 2024-04-23T21:21:40.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 19948 2024-04-23T21:21:40.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:40.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:40.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:40.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:40.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:40.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:40.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:40.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:40.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:40.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:40.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:40.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:40.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:40.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:40.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:40.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:40.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:40.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:40.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:40.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:40.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:40.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:40.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:40.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:40.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:40.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:40.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:40.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:40.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:40.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:40.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:40.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:40.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:40.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:40.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:40.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:40.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:40.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:40.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:40.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:40.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:40.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:40.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:40.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:40.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:40.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:40.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:40.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:40.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:40.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:40.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:40.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:41.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:41.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:41.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:41.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:41.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:41.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:41.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:41.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:41.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:41.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:41.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:41.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:41.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:41.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:41.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:41.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:41.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:41.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:41.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:41.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:41.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:41.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:41.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:41.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:41.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:41.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:41.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:41.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:41.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:41.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:41.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:41.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:41.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:41.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:41.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=17, tag_tid=2, entry_tid=20273], [object_number=16, tag_tid=2, entry_tid=20272], [object_number=19, tag_tid=2, entry_tid=20271], [object_number=18, tag_tid=2, entry_tid=20270]]], state=connected] 2024-04-23T21:21:41.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:41.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:41.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:41.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:41.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=17, tag_tid=2, entry_tid=20273], [object_number=16, tag_tid=2, entry_tid=20272], [object_number=19, tag_tid=2, entry_tid=20271], [object_number=18, tag_tid=2, entry_tid=20270]]], state=connected] 2024-04-23T21:21:41.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:41.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=17, tag_tid=2, entry_tid=20273' 2024-04-23T21:21:41.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=17, tag_tid=2, entry_tid=20273' 2024-04-23T21:21:41.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=17, tag_tid=2, entry_tid=20273' '!=' 'object_number=16, tag_tid=2, entry_tid=19948' 2024-04-23T21:21:41.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:41.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=17, tag_tid=2, entry_tid=20273' = 'object_number=16, tag_tid=2, entry_tid=19948' 2024-04-23T21:21:41.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=17, tag_tid=2, entry_tid=20273' 2024-04-23T21:21:41.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:41.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:41.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:41.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:41.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=17, tag_tid=2, entry_tid=20273' 2024-04-23T21:21:41.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:41.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:41.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:41.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:41.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:41.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:41.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:41.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=17, tag_tid=2, entry_tid=20273' 2024-04-23T21:21:41.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:41.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:41.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=20273 2024-04-23T21:21:41.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 20273 2024-04-23T21:21:41.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:41.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:41.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:41.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:41.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:41.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:41.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:41.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:41.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:41.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:41.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:41.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:41.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:41.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:41.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:41.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:41.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:41.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:41.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:41.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:41.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:41.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:41.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:41.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:41.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:41.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:41.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:41.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:41.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:41.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:41.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:41.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:41.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:41.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:41.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:41.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:41.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:41.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:41.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:41.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:41.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:41.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:41.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:41.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:41.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:41.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:41.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:41.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:41.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:41.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:41.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:41.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:41.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:41.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:41.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:41.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:41.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:41.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:41.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:41.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:41.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:41.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:41.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:41.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:41.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:41.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:41.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:41.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:41.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:41.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:41.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:41.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:41.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:41.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:41.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:41.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:41.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:41.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:41.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:41.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:41.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:41.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:41.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:41.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:41.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=19, tag_tid=2, entry_tid=20575], [object_number=18, tag_tid=2, entry_tid=20574], [object_number=17, tag_tid=2, entry_tid=20573], [object_number=16, tag_tid=2, entry_tid=20572]]], state=connected] 2024-04-23T21:21:41.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:41.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:41.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:41.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:41.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:41.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:41.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:41.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:41.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:41.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:41.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:41.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:41.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:41.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:41.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:41.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:41.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:41.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:41.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:41.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:41.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:41.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=19, tag_tid=2, entry_tid=20575], [object_number=18, tag_tid=2, entry_tid=20574], [object_number=17, tag_tid=2, entry_tid=20573], [object_number=16, tag_tid=2, entry_tid=20572]]], state=connected] 2024-04-23T21:21:41.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:41.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=19, tag_tid=2, entry_tid=20575' 2024-04-23T21:21:41.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=19, tag_tid=2, entry_tid=20575' 2024-04-23T21:21:41.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=19, tag_tid=2, entry_tid=20575' '!=' 'object_number=17, tag_tid=2, entry_tid=20273' 2024-04-23T21:21:41.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:41.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=19, tag_tid=2, entry_tid=20575' = 'object_number=17, tag_tid=2, entry_tid=20273' 2024-04-23T21:21:41.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=19, tag_tid=2, entry_tid=20575' 2024-04-23T21:21:41.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:41.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:41.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:41.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:41.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=19, tag_tid=2, entry_tid=20575' 2024-04-23T21:21:41.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:41.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:41.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:41.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:41.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:41.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:41.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:41.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=19, tag_tid=2, entry_tid=20575' 2024-04-23T21:21:41.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:41.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:41.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=20575 2024-04-23T21:21:41.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 20575 2024-04-23T21:21:41.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:41.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:41.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:42.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:42.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:42.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:42.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:42.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:42.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:42.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:42.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:42.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:42.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:42.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:42.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:42.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:42.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:42.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:42.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:42.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:42.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:42.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:42.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:42.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:42.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:42.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:42.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:42.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:42.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:42.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:42.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:42.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:42.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:42.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:42.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:42.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:42.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:42.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:42.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:42.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:42.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:42.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:42.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:42.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:42.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:42.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:42.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:42.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:42.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:42.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:42.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:42.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:42.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:42.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:42.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:42.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:42.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:42.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:42.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:42.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:42.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:42.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:42.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:42.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:42.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:42.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:42.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:42.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:42.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:42.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:42.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:42.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:42.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:42.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:42.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:42.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:42.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:42.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:42.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:42.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:42.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:42.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:42.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:42.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:42.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:42.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:42.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:42.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:42.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:42.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:42.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:42.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:42.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:42.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:42.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:42.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:42.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:42.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:42.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:42.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:42.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:42.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:42.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:42.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:42.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:42.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:42.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:42.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:42.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:42.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:42.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:42.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:42.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:42.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:42.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:42.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:42.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:42.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:42.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:42.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:42.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:42.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:42.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:42.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:42.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:42.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:42.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:42.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:42.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:42.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:42.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:42.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:42.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:42.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:42.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:42.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:42.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=19, tag_tid=2, entry_tid=20783], [object_number=18, tag_tid=2, entry_tid=20782], [object_number=17, tag_tid=2, entry_tid=20781], [object_number=16, tag_tid=2, entry_tid=20780]]], state=connected] 2024-04-23T21:21:42.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:42.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:42.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:42.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:42.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:42.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:42.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:42.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:42.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:42.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:42.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:42.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:42.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:42.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:42.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:42.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:42.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:42.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:21:42.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:42.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:42.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:42.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=19, tag_tid=2, entry_tid=20783], [object_number=18, tag_tid=2, entry_tid=20782], [object_number=17, tag_tid=2, entry_tid=20781], [object_number=16, tag_tid=2, entry_tid=20780]]], state=connected] 2024-04-23T21:21:42.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:42.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=19, tag_tid=2, entry_tid=20783' 2024-04-23T21:21:42.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=19, tag_tid=2, entry_tid=20783' 2024-04-23T21:21:42.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=19, tag_tid=2, entry_tid=20783' '!=' 'object_number=19, tag_tid=2, entry_tid=20575' 2024-04-23T21:21:42.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:42.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=19, tag_tid=2, entry_tid=20783' = 'object_number=19, tag_tid=2, entry_tid=20575' 2024-04-23T21:21:42.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=19, tag_tid=2, entry_tid=20783' 2024-04-23T21:21:42.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:42.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:42.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:42.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:42.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=19, tag_tid=2, entry_tid=20783' 2024-04-23T21:21:42.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:42.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:42.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:42.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:42.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:42.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:42.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:42.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=19, tag_tid=2, entry_tid=20783' 2024-04-23T21:21:42.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:42.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:42.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=20783 2024-04-23T21:21:42.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 20783 2024-04-23T21:21:42.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:42.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:42.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:42.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:42.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:42.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:42.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:42.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:42.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:42.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:42.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:42.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:42.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:42.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:42.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:42.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:42.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:42.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:42.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:42.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:42.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:42.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:42.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:42.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:42.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:42.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:42.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:42.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:42.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:42.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:42.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:42.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:42.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:42.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:42.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:42.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:42.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:42.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:42.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:42.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:42.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:42.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:42.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:42.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:42.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:42.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:42.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:42.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:42.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:42.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:42.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:42.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:42.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:42.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:42.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:42.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:42.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:42.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:42.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:42.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:42.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:42.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:42.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:42.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:42.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:42.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:42.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:42.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:42.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:42.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:42.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:42.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:42.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:42.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:42.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:42.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:42.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:42.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:42.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:42.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:42.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:42.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:42.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:42.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:42.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:42.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:42.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:42.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:42.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:42.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:42.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:42.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:42.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:42.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:42.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:42.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:42.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:42.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:42.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:42.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:42.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:42.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:42.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:42.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:42.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:42.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:42.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:42.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:42.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:42.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:42.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:42.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:42.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:42.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:42.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:42.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:42.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:43.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:43.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:43.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:43.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:43.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:43.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:43.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 5 2024-04-23T21:21:43.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:43.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:43.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:43.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=23, tag_tid=2, entry_tid=21043], [object_number=22, tag_tid=2, entry_tid=21042], [object_number=21, tag_tid=2, entry_tid=21041], [object_number=20, tag_tid=2, entry_tid=21040]]], state=connected] 2024-04-23T21:21:43.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:43.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:43.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:43.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:43.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:43.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:43.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:43.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:43.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:43.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:43.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:43.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:43.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:43.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:43.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:43.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:43.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:43.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 5 2024-04-23T21:21:43.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:43.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:43.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:43.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=23, tag_tid=2, entry_tid=21043], [object_number=22, tag_tid=2, entry_tid=21042], [object_number=21, tag_tid=2, entry_tid=21041], [object_number=20, tag_tid=2, entry_tid=21040]]], state=connected] 2024-04-23T21:21:43.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:43.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=23, tag_tid=2, entry_tid=21043' 2024-04-23T21:21:43.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=23, tag_tid=2, entry_tid=21043' 2024-04-23T21:21:43.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=23, tag_tid=2, entry_tid=21043' '!=' 'object_number=19, tag_tid=2, entry_tid=20783' 2024-04-23T21:21:43.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:43.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=23, tag_tid=2, entry_tid=21043' = 'object_number=19, tag_tid=2, entry_tid=20783' 2024-04-23T21:21:43.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=23, tag_tid=2, entry_tid=21043' 2024-04-23T21:21:43.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:43.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:43.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:43.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:43.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=23, tag_tid=2, entry_tid=21043' 2024-04-23T21:21:43.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:43.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:43.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:43.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:43.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:43.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:43.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:43.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=23, tag_tid=2, entry_tid=21043' 2024-04-23T21:21:43.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:43.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:43.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=21043 2024-04-23T21:21:43.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 21043 2024-04-23T21:21:43.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:43.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:43.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:43.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:43.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:43.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:43.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:43.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:43.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:43.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:43.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:43.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:43.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:43.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:43.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:43.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:43.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:43.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:43.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:43.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:43.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:43.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:43.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:43.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:43.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:43.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:43.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:43.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:43.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:43.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:43.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:43.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:43.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:43.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:43.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:43.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:43.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:43.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:43.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:43.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:43.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:43.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:43.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:43.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:43.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:43.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:43.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:43.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:43.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:43.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:43.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:43.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:43.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:43.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:43.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:43.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:43.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:43.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:43.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:43.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:43.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:43.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:43.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:43.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:43.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:43.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:43.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:43.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:43.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:43.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:43.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:43.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:43.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:43.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:43.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:43.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:43.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:43.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:43.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:43.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:43.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:43.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:43.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:43.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:43.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:43.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:43.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:43.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:43.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:43.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:43.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:43.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:43.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:43.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:43.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:43.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:43.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:43.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:43.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:43.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:43.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:43.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:43.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:43.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:43.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:43.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:43.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:43.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:43.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:43.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:43.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:43.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:43.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:43.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:43.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:43.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:43.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:43.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:43.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:43.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:43.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:43.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:43.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:43.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:43.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:43.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:43.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:43.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:43.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:43.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:43.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:43.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:43.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:43.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:43.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:43.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:43.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:43.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:43.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 5 2024-04-23T21:21:43.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:43.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:43.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:43.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=23, tag_tid=2, entry_tid=21339], [object_number=22, tag_tid=2, entry_tid=21338], [object_number=21, tag_tid=2, entry_tid=21337], [object_number=20, tag_tid=2, entry_tid=21336]]], state=connected] 2024-04-23T21:21:43.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:43.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:43.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:43.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:43.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:43.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:43.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:43.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:43.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:43.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:43.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:43.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:43.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:43.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:43.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:43.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:43.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:43.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 5 2024-04-23T21:21:43.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:43.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:43.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:43.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=23, tag_tid=2, entry_tid=21339], [object_number=22, tag_tid=2, entry_tid=21338], [object_number=21, tag_tid=2, entry_tid=21337], [object_number=20, tag_tid=2, entry_tid=21336]]], state=connected] 2024-04-23T21:21:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:43.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=23, tag_tid=2, entry_tid=21339' 2024-04-23T21:21:43.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=23, tag_tid=2, entry_tid=21339' 2024-04-23T21:21:43.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=23, tag_tid=2, entry_tid=21339' '!=' 'object_number=23, tag_tid=2, entry_tid=21043' 2024-04-23T21:21:43.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:43.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=23, tag_tid=2, entry_tid=21339' = 'object_number=23, tag_tid=2, entry_tid=21043' 2024-04-23T21:21:43.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=23, tag_tid=2, entry_tid=21339' 2024-04-23T21:21:43.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:43.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:43.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:43.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:43.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=23, tag_tid=2, entry_tid=21339' 2024-04-23T21:21:43.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:43.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:43.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:43.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:43.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:43.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:43.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:43.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=23, tag_tid=2, entry_tid=21339' 2024-04-23T21:21:43.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:43.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:43.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=21339 2024-04-23T21:21:43.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 21339 2024-04-23T21:21:43.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:43.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:43.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:43.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:43.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:43.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:43.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:43.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:43.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:43.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:43.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:43.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:43.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:43.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:43.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:43.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:43.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:44.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:44.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:44.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:44.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:44.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:44.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:44.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:44.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:44.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:44.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:44.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:44.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:44.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:44.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:44.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:44.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:44.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:44.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:44.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:44.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:44.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:44.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:44.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:44.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:44.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:44.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:44.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:44.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:44.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:44.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:44.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:44.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:44.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:44.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:44.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:44.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:44.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:44.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:44.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:44.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:44.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:44.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:44.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:44.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:44.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:44.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:44.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:44.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:44.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:44.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:44.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:44.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:44.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:44.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:44.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:44.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:44.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:44.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:44.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:44.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:44.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:44.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:44.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:44.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:44.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:44.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:44.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:44.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:44.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:44.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:44.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:44.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:44.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:44.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:44.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:44.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:44.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:44.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:44.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:44.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:44.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:44.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:44.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:44.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:44.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:44.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:44.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 5 2024-04-23T21:21:44.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:44.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:44.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:44.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402], [object_number=21, tag_tid=2, entry_tid=21401], [object_number=20, tag_tid=2, entry_tid=21400]]], state=connected] 2024-04-23T21:21:44.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:44.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:44.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:44.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:44.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:44.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:44.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:44.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:44.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:44.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:44.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:44.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:44.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:44.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:44.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:44.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:44.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:44.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 5 2024-04-23T21:21:44.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:44.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:44.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:44.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402], [object_number=21, tag_tid=2, entry_tid=21401], [object_number=20, tag_tid=2, entry_tid=21400]]], state=connected] 2024-04-23T21:21:44.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:44.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=23, tag_tid=2, entry_tid=21403' 2024-04-23T21:21:44.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=23, tag_tid=2, entry_tid=21403' 2024-04-23T21:21:44.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=23, tag_tid=2, entry_tid=21403' '!=' 'object_number=23, tag_tid=2, entry_tid=21339' 2024-04-23T21:21:44.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:21:44.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=23, tag_tid=2, entry_tid=21403' = 'object_number=23, tag_tid=2, entry_tid=21339' 2024-04-23T21:21:44.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=23, tag_tid=2, entry_tid=21403' 2024-04-23T21:21:44.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:44.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:44.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:44.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:21:44.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=23, tag_tid=2, entry_tid=21403' 2024-04-23T21:21:44.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:21:44.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:44.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:21:44.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:44.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:44.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:44.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:21:44.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=23, tag_tid=2, entry_tid=21403' 2024-04-23T21:21:44.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:21:44.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:21:44.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=21403 2024-04-23T21:21:44.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 21403 2024-04-23T21:21:44.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:21:44.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:21:44.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:21:44.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:21:44.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:44.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:44.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:44.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:21:44.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:21:44.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:21:44.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:21:44.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:21:44.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:44.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:44.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:21:44.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:21:44.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:21:44.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:21:44.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:21:44.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:21:44.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:44.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:21:44.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:21:44.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:44.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:44.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:21:44.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:21:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:21:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:21:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:21:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:21:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:21:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:21:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:21:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:21:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:21:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:21:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:21:44.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:44.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:44.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:21:44.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:21:44.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:44.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:44.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:44.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:21:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:21:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:21:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:21:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:21:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:21:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:21:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:21:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:21:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:21:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:21:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:21:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:44.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:21:44.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:21:44.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:21:44.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:44.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:44.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:21:44.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:21:44.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:21:44.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:21:44.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:21:44.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:21:44.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:21:44.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:21:44.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:21:44.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:21:44.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:21:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:21:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:21:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:21:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:21:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:21:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:21:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:21:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:21:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:21:44.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:21:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:21:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:21:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:21:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:21:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:21:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:21:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:21:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:21:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:21:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:21:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:21:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:21:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:21:44.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:44.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:44.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:21:44.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:21:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:21:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:21:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:21:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:44.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:44.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 5 2024-04-23T21:21:44.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:44.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:44.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:44.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:44.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:44.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:44.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:21:44.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:44.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:44.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:44.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:21:44.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:21:44.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:21:44.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:21:44.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:21:44.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:21:44.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:21:44.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:21:44.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:44.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:21:44.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:44.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 5 2024-04-23T21:21:44.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 5 2024-04-23T21:21:44.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:21:44.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:44.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=21, tag_tid=3, entry_tid=1], [object_number=20, tag_tid=3, entry_tid=0], [object_number=23, tag_tid=2, entry_tid=21403], [object_number=22, tag_tid=2, entry_tid=21402]]], state=connected] 2024-04-23T21:21:44.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:21:44.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:44.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=21, tag_tid=3, entry_tid=1' -a 'object_number=21, tag_tid=3, entry_tid=1' = 'object_number=21, tag_tid=3, entry_tid=1' 2024-04-23T21:21:44.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:44.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror test snap1 2024-04-23T21:21:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:21:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap1 2024-04-23T21:21:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:21:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:21:44.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:21:45.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/test@snap1 2024-04-23T21:21:45.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'test': 2024-04-23T21:21:45.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:21:45.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:21:45.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:21:45.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10aff12cc611 2024-04-23T21:21:45.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10aff12cc611 2024-04-23T21:21:45.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:21:45.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:21:45.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:21:45.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:21:45.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:20:44 2024 2024-04-23T21:21:45.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:20:44 2024 2024-04-23T21:21:45.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:20:44 2024 2024-04-23T21:21:45.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:21:45.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10aff12cc611 2024-04-23T21:21:45.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:21:45.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:21:45.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:21:45.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:21:45.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:45.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:21:45.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap1 2024-04-23T21:21:45.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:45.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:45.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap1 2024-04-23T21:21:45.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:21:45.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:21:45.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:21:45.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:21:45.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:21:46.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:21:46.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:21:46.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:21:46.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:21:47.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:21:47.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:21:47.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ clean_snap_name=snap1-clean 2024-04-23T21:21:47.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap1-clean 2024-04-23T21:21:47.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:21:47.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:47.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:47.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap1-clean 2024-04-23T21:21:47.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap1-clean --debug-rbd=20 --debug-journaler=20 2024-04-23T21:21:47.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:21:47.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror test 2024-04-23T21:21:47.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:21:47.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:21:47.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:21:47.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:21:47.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=13 2024-04-23T21:21:47.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:21:47.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2024-04-23T21:21:51.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 104972288 bytes 2024-04-23T21:21:52.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 142390272 bytes 2024-04-23T21:21:54.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 175787008 bytes 2024-04-23T21:21:55.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 237636608 bytes 2024-04-23T21:21:57.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 267175936 bytes 2024-04-23T21:21:57.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:600 IOs, 267771904 bytes 2024-04-23T21:21:59.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:700 IOs, 303078400 bytes 2024-04-23T21:22:00.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:22:00.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:22:00.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:22:00.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:22:00.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2024-04-23T21:22:00.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:22:00.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:22:00.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:22:00.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:22:00.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:22:00.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:22:00.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:22:00.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:22:01.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2024-04-23T21:22:01.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:22:01.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:22:01.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:22:01.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:22:01.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:22:01.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:22:01.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:22:01.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:22:01.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2024-04-23T21:22:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='test: 2024-04-23T21:22:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: global_id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:22:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+replaying 2024-04-23T21:22:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: replaying, {"bytes_per_second":6498912.67,"entries_behind_primary":621,"entries_per_second":404.6,"non_primary_position":{"entry_tid":20783,"object_number":19,"tag_tid":2},"primary_position":{"entry_tid":1,"object_number":21,"tag_tid":3},"seconds_until_synced":1} 2024-04-23T21:22:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: service: mirror.2 on smithi150 2024-04-23T21:22:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:21:42 2024-04-23T21:22:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: peer_sites: 2024-04-23T21:22:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: name: cluster2 2024-04-23T21:22:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+stopped 2024-04-23T21:22:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: local image is primary 2024-04-23T21:22:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:21:42' 2024-04-23T21:22:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ state: .*up+replaying' 2024-04-23T21:22:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: state: up+replaying 2024-04-23T21:22:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ description: .*' 2024-04-23T21:22:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: description: replaying, {"bytes_per_second":6498912.67,"entries_behind_primary":621,"entries_per_second":404.6,"non_primary_position":{"entry_tid":20783,"object_number":19,"tag_tid":2},"primary_position":{"entry_tid":1,"object_number":21,"tag_tid":3},"seconds_until_synced":1} 2024-04-23T21:22:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:22:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:22:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^up+' 2024-04-23T21:22:01.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:22:01.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep 'service: *mirror..* on ' 2024-04-23T21:22:01.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: service: mirror.2 on smithi150 2024-04-23T21:22:01.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update 2024-04-23T21:22:01.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-04-23T21:22:01.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:21:42' 2024-04-23T21:22:01.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2024-04-23 21:21:42' 2024-04-23T21:22:01.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:22:01.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:22:01.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:22:01.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:22:01.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'prev_last_update=2024-04-23 21:21:42' 2024-04-23T21:22:01.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:22:01.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:22:01.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-04-23T21:22:01.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='OKOKOK142174271mirror.2smithi15019.0.0-3212-g8254cb5atrueOK42184274mirror.1smithi15019.0.0-3212-g8254cb5afalseOK42224277mirror.3smithi15019.0.0-3212-g8254cb5afalseOK42624319mirror.0smithi15019.0.0-3212-g8254cb5afalseOKteste56bd005-f8a9-476d-a8fc-713b4e8cd286up+replayingreplaying, {"bytes_per_second":6498912.67,"entries_behind_primary":621,"entries_per_second":404.6,"non_primary_position":{"entry_tid":20783,"object_number":19,"tag_tid":2},"primary_position":{"entry_tid":1,"object_number":21,"tag_tid":3},"seconds_until_synced":1}42174271mirror.2smithi1502024-04-23 21:21:42cluster267253cce-c621-4f62-a184-da88591999d7up+stoppedlocal image is primary2024-04-23 21:21:42' 2024-04-23T21:22:01.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update state 2024-04-23T21:22:01.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2024-04-23T21:22:01.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:21:42' 2024-04-23T21:22:01.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2024-04-23T21:22:01.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ state=up+replaying 2024-04-23T21:22:01.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:22:01.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep up+replaying 2024-04-23T21:22:01.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:22:01.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:22:01.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:22:01.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap2 2024-04-23T21:22:01.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap2 2024-04-23T21:22:01.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:22:01.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:22:01.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:22:01.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap2 2024-04-23T21:22:01.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap2 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:22:16.716 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9760710236899203) > weight (0.25) 2024-04-23T21:22:16.716 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.46811280448012904) > weight (0.25) 2024-04-23T21:22:16.716 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2024-04-23T21:22:16.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2024-04-23T21:22:16.717 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2024-04-23T21:22:16.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:22:16.718+0000 7fb8b6eca640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 35417) UID: 1000 2024-04-23T21:22:21.717 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2024-04-23T21:22:21.717 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:22:27.819 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2024-04-23T21:22:27.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2024-04-23T21:22:27.819 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2024-04-23T21:22:27.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2024-04-23T21:22:27.823 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2024-04-23T21:22:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror test 2024-04-23T21:22:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:22:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:22:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:22:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2024-04-23T21:22:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:22:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:22:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:22:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:22:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:22:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:22:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:22:32.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror test started 2024-04-23T21:22:32.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:22:32.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:22:32.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:22:32.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:22:32.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:22:32.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:22:32.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2024-04-23T21:22:32.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:22:32.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:22:32.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:32.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:22:32.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:22:32.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:22:32.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:22:32.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:22:32.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:22:32.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:22:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:22:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:22:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:22:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:22:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:22:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:32.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:22:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:22:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:22:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:22:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:22:32.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:22:32.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:32.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:32.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:32.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:22:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:22:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:22:32.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:22:32.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:32.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:32.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:32.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:22:32.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:32.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:32.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:32.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:32.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:22:32.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:22:32.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:32.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:32.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:22:32.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:22:32.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:32.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:32.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:32.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:22:32.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:32.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:32.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:32.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:22:32.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:22:32.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:22:32.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:22:32.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:32.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:32.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:22:32.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:22:32.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:32.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:32.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:32.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:22:32.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:32.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:32.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:32.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:22:32.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:22:32.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:22:32.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:32.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:32.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:32.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:22:32.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:32.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:32.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:32.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:32.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2024-04-23T21:22:32.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:22:32.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:32.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:32.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:22:32.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:22:32.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:32.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:32.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:32.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:22:32.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:22:32.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:32.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:32.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:22:32.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:22:32.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:22:32.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:22:32.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:32.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:32.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:22:32.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:22:32.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:32.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:32.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:32.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:22:32.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:22:32.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:32.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:32.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:22:32.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:22:32.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:22:32.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:22:32.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:22:32.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:22:32.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2024-04-23T21:22:33.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:33.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:33.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:33.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:33.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2024-04-23T21:22:33.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:22:33.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:33.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:22:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:22:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:22:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:22:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:22:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:22:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:22:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:22:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:33.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:22:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:22:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:22:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:22:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:22:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:22:33.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:22:33.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:22:33.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:22:33.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:22:33.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2024-04-23T21:22:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:33.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:33.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:33.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:22:33.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 1 2024-04-23T21:22:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:22:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:22:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:22:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:22:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:22:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:22:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:34.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:34.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:22:34.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:34.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:34.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:34.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:22:34.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:22:34.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:22:34.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:22:34.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:34.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:34.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:22:34.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:22:34.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:34.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:34.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:34.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:22:34.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:34.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:34.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:34.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:22:34.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:22:34.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:22:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:22:34.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:34.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:34.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:34.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:22:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:22:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:22:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:22:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:22:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:34.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:34.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:34.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:22:34.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:22:34.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:22:34.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:22:34.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:34.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:34.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:22:34.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:22:34.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:34.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:34.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:34.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:22:34.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:34.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:34.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:34.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:22:34.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:22:34.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:22:34.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:34.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:34.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:34.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:22:34.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:34.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:34.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:34.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:34.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2024-04-23T21:22:34.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:22:34.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:34.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:34.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:22:34.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:22:34.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:34.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:34.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:34.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:22:34.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:22:34.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:34.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:34.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:22:34.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:22:34.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:22:34.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:22:34.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:22:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:22:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:22:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:22:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:22:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:22:34.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:22:34.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:22:34.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:22:34.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:22:34.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2024-04-23T21:22:34.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:34.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:34.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2024-04-23T21:22:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:22:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:22:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:22:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:34.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:22:34.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:22:34.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:34.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:34.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:22:34.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:22:34.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:22:34.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:22:34.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:34.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:34.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:22:34.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:22:34.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:34.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:34.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:34.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:22:34.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:22:34.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:34.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:34.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:22:34.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:22:34.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:22:34.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:22:34.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:22:34.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:22:34.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2024-04-23T21:22:34.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:34.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:34.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:34.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:22:34.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 2 2024-04-23T21:22:36.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:22:36.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:22:36.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:36.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:22:36.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:22:36.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:36.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:36.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:22:36.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:22:36.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:36.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:36.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:36.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:22:36.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:36.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:22:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:22:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:22:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:22:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:22:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:22:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:22:36.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:22:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:22:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:22:36.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:36.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:36.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:36.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:22:36.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:36.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:36.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:36.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:36.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:22:36.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:22:36.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:36.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:36.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:22:36.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:22:36.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:36.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:36.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:22:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:22:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:22:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:22:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:22:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:22:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:22:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:36.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:22:36.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:36.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:36.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:36.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:22:36.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:22:36.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:22:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:22:36.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:36.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:36.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:36.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:36.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2024-04-23T21:22:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:22:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:22:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:22:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:22:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:22:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:22:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:22:36.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:22:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:22:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:22:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:22:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:22:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:22:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:22:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:22:36.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:22:36.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:22:36.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:22:36.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:22:36.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2024-04-23T21:22:36.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:36.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:36.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:36.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:36.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2024-04-23T21:22:36.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:22:36.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:36.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:36.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:22:36.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:22:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:22:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:22:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:22:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:22:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:22:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:22:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:22:36.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:22:36.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:36.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:36.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:36.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:22:36.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:22:36.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:36.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:36.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:22:36.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:22:36.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:22:36.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:22:36.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:22:36.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:22:36.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2024-04-23T21:22:36.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:36.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:36.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:36.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:22:36.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 4 2024-04-23T21:22:40.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:22:40.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:22:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:22:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:22:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:22:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:22:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:22:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:22:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:22:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:22:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:22:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:22:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:22:40.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:40.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:40.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:40.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:22:40.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:40.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:40.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:40.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:22:40.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:22:40.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:22:40.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:40.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:40.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:40.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:22:41.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:41.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:41.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:41.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:41.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:22:41.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:22:41.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:41.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:41.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:22:41.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:22:41.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:41.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:41.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:41.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:22:41.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:41.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:41.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:41.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:22:41.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:22:41.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:22:41.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:22:41.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:41.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:41.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:22:41.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:22:41.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:41.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:41.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:41.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:22:41.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:41.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:41.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:41.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:22:41.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:22:41.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:22:41.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:41.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:41.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:41.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:22:41.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:41.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:41.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2024-04-23T21:22:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:22:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:22:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:22:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:22:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:22:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:41.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:22:41.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:22:41.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:22:41.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:22:41.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:41.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:41.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:22:41.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:22:41.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:41.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:41.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:41.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:22:41.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:22:41.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:41.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:41.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:22:41.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:22:41.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:22:41.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:22:41.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:22:41.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:22:41.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2024-04-23T21:22:41.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:41.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:41.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:41.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:41.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2024-04-23T21:22:41.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:22:41.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:22:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:22:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:22:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:22:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:22:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:22:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:22:41.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:22:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:22:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:22:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:22:41.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:22:41.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:41.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:41.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:22:41.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:22:41.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:22:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:22:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:22:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:22:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2024-04-23T21:22:41.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:41.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:41.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:22:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 8 2024-04-23T21:22:49.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:22:49.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:22:49.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:49.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:22:49.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:22:49.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:49.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:49.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:22:49.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:22:49.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:49.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:49.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:49.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:22:49.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:49.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:49.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:49.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:22:49.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:22:49.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:22:49.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:22:49.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:49.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:49.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:22:49.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:22:49.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:49.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:49.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:49.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:22:49.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:49.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:49.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:49.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:22:49.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:22:49.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:22:49.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:49.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:49.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:49.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:22:49.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:49.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:49.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:49.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:49.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:22:49.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:22:49.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:49.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:49.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:22:49.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:22:49.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:22:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:22:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:22:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:22:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:22:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:22:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:22:49.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:49.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:49.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:49.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:22:49.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:49.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:49.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:49.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:22:49.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:22:49.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:22:49.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:49.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:49.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:49.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:22:49.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:49.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:49.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:49.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:49.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2024-04-23T21:22:49.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:22:49.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:49.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:49.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:22:49.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:22:49.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:49.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:49.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:49.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:22:49.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:22:49.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:49.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:49.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:22:49.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:22:49.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:22:49.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:22:49.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:49.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:49.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:22:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:22:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:22:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:22:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:22:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:22:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:22:49.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:22:49.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:22:49.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:22:49.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2024-04-23T21:22:49.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:49.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:49.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:49.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:49.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2024-04-23T21:22:49.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:22:49.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:49.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:49.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:22:49.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:22:49.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:49.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:49.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:49.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:22:49.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:22:49.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:49.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:49.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:22:49.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:22:49.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:22:49.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:22:49.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:49.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:49.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:22:49.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:22:49.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:49.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:49.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:49.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:22:49.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:22:49.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:49.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:49.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:22:49.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:22:49.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:22:49.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:22:49.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:22:49.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:22:49.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2024-04-23T21:22:49.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:49.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:49.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:49.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:22:49.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 8 2024-04-23T21:22:57.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:22:57.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:22:57.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:22:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:22:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:22:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:22:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:22:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:22:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:22:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:22:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:22:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:22:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:22:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:57.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:22:57.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:57.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:57.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:57.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:22:57.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:22:57.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:22:57.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:57.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:57.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:57.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:22:57.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:57.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:22:57.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:22:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:22:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:22:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:22:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:22:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:22:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:22:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:57.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:22:57.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:57.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:22:57.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:22:57.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:22:57.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:22:57.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:22:57.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:22:57.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:22:57.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:22:57.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:22:57.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:22:57.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:22:57.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:22:57.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:57.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:22:57.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:22:57.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:22:57.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:22:57.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:22:57.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:57.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:57.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:57.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:22:57.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:22:57.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:22:57.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:22:57.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:22:57.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:22:57.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:22:57.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:22:57.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:22:57.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:22:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:22:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:22:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:22:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:22:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:22:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:22:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:22:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:22:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:22:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:22:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:22:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:22:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:22:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:22:57.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:22:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:22:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:22:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:22:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:22:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:22:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:22:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:22:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:22:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:22:58.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:22:58.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:22:58.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:22:58.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:22:58.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:22:58.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:22:58.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:22:58.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:22:58.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:58.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:22:58.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:22:58.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:22:58.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:22:58.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:22:58.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:22:58.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:22:58.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:22:58.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:22:58.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:22:58.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:58.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:22:58.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:58.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:22:58.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:22:58.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:22:58.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:22:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:22:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:22:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:22:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:22:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:22:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:22:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:22:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:22:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:22:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:22:58.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:58.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:58.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:58.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:22:58.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:58.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:22:58.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:22:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:22:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:22:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:22:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:22:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:22:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:22:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:22:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:22:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:22:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:22:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:22:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:22:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:22:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:22:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:22:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:58.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:22:58.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:22:58.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:22:58.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:58.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:58.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:22:58.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:58.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:22:58.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:22:58.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:22:58.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:22:58.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:22:58.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:58.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:58.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:58.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:22:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:22:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:22:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:22:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:22:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:22:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:22:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:22:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:22:58.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:22:58.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:22:58.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:22:58.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:22:58.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:22:58.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:58.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:22:58.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:58.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 7 2024-04-23T21:22:58.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:22:58.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:22:58.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:22:58.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=28, tag_tid=5, entry_tid=11140], [object_number=31, tag_tid=5, entry_tid=11139], [object_number=30, tag_tid=5, entry_tid=11138], [object_number=29, tag_tid=5, entry_tid=11137]]], state=connected] 2024-04-23T21:22:58.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:58.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:22:58.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:22:58.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:22:58.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:22:58.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:22:58.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:22:58.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:22:58.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:22:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:22:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:22:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:22:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:22:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:22:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:22:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:58.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 7 2024-04-23T21:22:58.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:22:58.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:22:58.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:22:58.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=28, tag_tid=5, entry_tid=11140], [object_number=31, tag_tid=5, entry_tid=11139], [object_number=30, tag_tid=5, entry_tid=11138], [object_number=29, tag_tid=5, entry_tid=11137]]], state=connected] 2024-04-23T21:22:58.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:58.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=28, tag_tid=5, entry_tid=11140' 2024-04-23T21:22:58.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=28, tag_tid=5, entry_tid=11140' 2024-04-23T21:22:58.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=28, tag_tid=5, entry_tid=11140' '!=' '' 2024-04-23T21:22:58.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:22:58.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=28, tag_tid=5, entry_tid=11140' = '' 2024-04-23T21:22:58.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=28, tag_tid=5, entry_tid=11140' 2024-04-23T21:22:58.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:22:58.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:22:58.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:22:58.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:22:58.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=28, tag_tid=5, entry_tid=11140' 2024-04-23T21:22:58.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:22:58.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:22:58.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:22:58.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:22:58.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:22:58.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:22:58.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:22:58.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=28, tag_tid=5, entry_tid=11140' 2024-04-23T21:22:58.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:22:58.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:22:58.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=11140 2024-04-23T21:22:58.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 11140 2024-04-23T21:22:58.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:22:58.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:22:58.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:22:58.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:22:58.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:22:58.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:22:58.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:22:58.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:22:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:22:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:22:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:22:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:22:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:22:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:22:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:22:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:22:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:22:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:22:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:22:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:22:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:22:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:22:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:22:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:22:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:22:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:22:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:22:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:22:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:22:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:22:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:22:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:22:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:22:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:22:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:22:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:22:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:22:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:22:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:22:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:22:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:22:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:22:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:22:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:22:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:22:58.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:58.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:58.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:58.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:22:58.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:58.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:22:58.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:58.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:58.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:22:58.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:22:58.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:22:58.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:22:58.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:22:58.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:22:58.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:22:58.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:22:58.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:22:58.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:22:58.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:58.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:22:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:22:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:22:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:22:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:22:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:22:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:22:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:22:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:22:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:22:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:58.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:22:58.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:22:58.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:22:58.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:22:58.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:22:58.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:58.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:58.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:58.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:22:58.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:22:58.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:22:58.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:22:58.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:22:58.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:22:58.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:22:58.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:22:58.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:22:58.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:22:58.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:22:58.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:22:58.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:22:58.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:22:58.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:58.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:22:58.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:58.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 7 2024-04-23T21:22:58.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:22:58.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:22:58.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:22:58.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=28, tag_tid=5, entry_tid=11308], [object_number=31, tag_tid=5, entry_tid=11307], [object_number=30, tag_tid=5, entry_tid=11306], [object_number=29, tag_tid=5, entry_tid=11305]]], state=connected] 2024-04-23T21:22:58.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:58.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:22:58.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:22:58.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:22:58.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:22:58.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:22:58.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:22:58.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:22:58.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:22:58.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:22:58.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:22:58.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:22:58.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:22:58.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:22:58.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:58.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:22:58.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:58.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 7 2024-04-23T21:22:58.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:22:58.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:22:58.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:22:58.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=28, tag_tid=5, entry_tid=11308], [object_number=31, tag_tid=5, entry_tid=11307], [object_number=30, tag_tid=5, entry_tid=11306], [object_number=29, tag_tid=5, entry_tid=11305]]], state=connected] 2024-04-23T21:22:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:58.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=28, tag_tid=5, entry_tid=11308' 2024-04-23T21:22:58.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=28, tag_tid=5, entry_tid=11308' 2024-04-23T21:22:58.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=28, tag_tid=5, entry_tid=11308' '!=' 'object_number=28, tag_tid=5, entry_tid=11140' 2024-04-23T21:22:58.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:22:58.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=28, tag_tid=5, entry_tid=11308' = 'object_number=28, tag_tid=5, entry_tid=11140' 2024-04-23T21:22:58.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=28, tag_tid=5, entry_tid=11308' 2024-04-23T21:22:58.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:22:58.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:22:58.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:22:58.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:22:58.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=28, tag_tid=5, entry_tid=11308' 2024-04-23T21:22:58.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:22:58.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:22:58.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:22:58.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:22:58.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:22:58.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:22:58.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:22:58.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=28, tag_tid=5, entry_tid=11308' 2024-04-23T21:22:58.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:22:58.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:22:58.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=11308 2024-04-23T21:22:58.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 11308 2024-04-23T21:22:58.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:22:58.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:22:58.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:22:59.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:22:59.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:22:59.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:22:59.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:22:59.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:22:59.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:22:59.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:22:59.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:22:59.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:22:59.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:22:59.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:22:59.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:22:59.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:22:59.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:22:59.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:22:59.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:22:59.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:22:59.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:59.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:22:59.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:22:59.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:22:59.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:22:59.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:22:59.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:22:59.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:22:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:22:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:22:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:22:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:22:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:22:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:22:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:22:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:22:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:22:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:22:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:22:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:22:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:22:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:59.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:22:59.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:22:59.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:22:59.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:22:59.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:22:59.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:59.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:59.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:59.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:22:59.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:59.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:22:59.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:59.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:59.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:22:59.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:22:59.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:22:59.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:22:59.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:22:59.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:22:59.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:22:59.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:22:59.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:22:59.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:22:59.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:59.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:22:59.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:59.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:22:59.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:22:59.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:22:59.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:22:59.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:22:59.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:59.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:22:59.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:22:59.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:22:59.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:59.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:59.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:22:59.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:59.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:22:59.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:22:59.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:22:59.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:22:59.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:22:59.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:59.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:59.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:59.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:22:59.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:22:59.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:22:59.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:22:59.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:22:59.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:22:59.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:22:59.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:22:59.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:22:59.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:22:59.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:22:59.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:22:59.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:22:59.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:22:59.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:22:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:59.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:22:59.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:22:59.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:22:59.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:22:59.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=32, tag_tid=5, entry_tid=11416], [object_number=35, tag_tid=5, entry_tid=11415], [object_number=34, tag_tid=5, entry_tid=11414], [object_number=33, tag_tid=5, entry_tid=11413]]], state=connected] 2024-04-23T21:22:59.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:22:59.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:22:59.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:22:59.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:22:59.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:22:59.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:22:59.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:22:59.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:22:59.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:22:59.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:22:59.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:22:59.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:22:59.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:22:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:22:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:59.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:22:59.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:22:59.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:22:59.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:22:59.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=32, tag_tid=5, entry_tid=11416], [object_number=35, tag_tid=5, entry_tid=11415], [object_number=34, tag_tid=5, entry_tid=11414], [object_number=33, tag_tid=5, entry_tid=11413]]], state=connected] 2024-04-23T21:22:59.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:59.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=32, tag_tid=5, entry_tid=11416' 2024-04-23T21:22:59.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=32, tag_tid=5, entry_tid=11416' 2024-04-23T21:22:59.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=32, tag_tid=5, entry_tid=11416' '!=' 'object_number=28, tag_tid=5, entry_tid=11308' 2024-04-23T21:22:59.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:22:59.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=32, tag_tid=5, entry_tid=11416' = 'object_number=28, tag_tid=5, entry_tid=11308' 2024-04-23T21:22:59.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=32, tag_tid=5, entry_tid=11416' 2024-04-23T21:22:59.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:22:59.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:22:59.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:22:59.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:22:59.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=32, tag_tid=5, entry_tid=11416' 2024-04-23T21:22:59.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:22:59.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:22:59.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:22:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:22:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:22:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:22:59.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:22:59.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=32, tag_tid=5, entry_tid=11416' 2024-04-23T21:22:59.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:22:59.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:22:59.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=11416 2024-04-23T21:22:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 11416 2024-04-23T21:22:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:22:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:22:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:22:59.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:22:59.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:22:59.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:22:59.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:22:59.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:22:59.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:22:59.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:22:59.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:22:59.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:22:59.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:22:59.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:22:59.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:22:59.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:22:59.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:22:59.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:22:59.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:22:59.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:22:59.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:59.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:22:59.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:22:59.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:22:59.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:22:59.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:22:59.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:22:59.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:22:59.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:22:59.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:22:59.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:22:59.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:59.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:22:59.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:59.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:22:59.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:22:59.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:22:59.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:22:59.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:22:59.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:59.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:22:59.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:22:59.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:22:59.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:59.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:59.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:22:59.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:22:59.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:22:59.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:22:59.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:22:59.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:22:59.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:22:59.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:59.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:59.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:22:59.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:22:59.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:22:59.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:22:59.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:22:59.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:22:59.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:22:59.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:22:59.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:22:59.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:22:59.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:22:59.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:22:59.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:22:59.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:22:59.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:22:59.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:22:59.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:22:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:22:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:22:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:22:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:22:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:22:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:22:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:22:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:22:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:22:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:22:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:22:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:22:59.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:22:59.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:22:59.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:22:59.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:22:59.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:22:59.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:22:59.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:59.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:59.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:22:59.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:22:59.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:22:59.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:22:59.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:22:59.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:22:59.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:22:59.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:22:59.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:22:59.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:22:59.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:22:59.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:22:59.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:22:59.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:22:59.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:22:59.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:59.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:22:59.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:59.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:22:59.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:22:59.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:22:59.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:22:59.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=34, tag_tid=5, entry_tid=11610], [object_number=33, tag_tid=5, entry_tid=11609], [object_number=32, tag_tid=5, entry_tid=11608], [object_number=35, tag_tid=5, entry_tid=11607]]], state=connected] 2024-04-23T21:22:59.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:59.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:22:59.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:22:59.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:22:59.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:22:59.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:22:59.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:22:59.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:22:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:22:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:22:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:22:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:22:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:22:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:22:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:59.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:22:59.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:59.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:22:59.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:22:59.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:22:59.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:22:59.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=34, tag_tid=5, entry_tid=11610], [object_number=33, tag_tid=5, entry_tid=11609], [object_number=32, tag_tid=5, entry_tid=11608], [object_number=35, tag_tid=5, entry_tid=11607]]], state=connected] 2024-04-23T21:22:59.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:22:59.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=34, tag_tid=5, entry_tid=11610' 2024-04-23T21:22:59.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=34, tag_tid=5, entry_tid=11610' 2024-04-23T21:22:59.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=34, tag_tid=5, entry_tid=11610' '!=' 'object_number=32, tag_tid=5, entry_tid=11416' 2024-04-23T21:22:59.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:22:59.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=34, tag_tid=5, entry_tid=11610' = 'object_number=32, tag_tid=5, entry_tid=11416' 2024-04-23T21:22:59.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=34, tag_tid=5, entry_tid=11610' 2024-04-23T21:22:59.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:22:59.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:22:59.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:22:59.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:22:59.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=34, tag_tid=5, entry_tid=11610' 2024-04-23T21:22:59.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:22:59.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:22:59.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:22:59.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:22:59.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:22:59.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:22:59.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:22:59.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=34, tag_tid=5, entry_tid=11610' 2024-04-23T21:22:59.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:22:59.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:22:59.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=11610 2024-04-23T21:22:59.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 11610 2024-04-23T21:22:59.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:22:59.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:22:59.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:00.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:00.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:00.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:00.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:00.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:00.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:00.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:00.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:00.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:00.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:00.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:00.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:00.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:00.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:00.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:00.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:00.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:00.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:00.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:00.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:00.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:00.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:00.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:00.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:00.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:00.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:00.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:00.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:00.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:00.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:00.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:00.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:00.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:00.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:00.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:00.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:00.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:00.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:00.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:00.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:00.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:00.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:00.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:00.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:00.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:00.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:00.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:00.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:00.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:00.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:00.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:00.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:00.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:00.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:00.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:00.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:00.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:00.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:00.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:00.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:00.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:00.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:00.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:00.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:00.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:00.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:00.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:00.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:00.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:00.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:00.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:00.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:00.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:00.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:00.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:00.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:00.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=33, tag_tid=5, entry_tid=11825], [object_number=32, tag_tid=5, entry_tid=11824], [object_number=35, tag_tid=5, entry_tid=11823], [object_number=34, tag_tid=5, entry_tid=11822]]], state=connected] 2024-04-23T21:23:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:00.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:00.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:00.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:00.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:00.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:00.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:00.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:00.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:00.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:00.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:00.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:00.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:00.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:00.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:00.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:00.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:00.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:00.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:00.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:00.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:00.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=33, tag_tid=5, entry_tid=11825], [object_number=32, tag_tid=5, entry_tid=11824], [object_number=35, tag_tid=5, entry_tid=11823], [object_number=34, tag_tid=5, entry_tid=11822]]], state=connected] 2024-04-23T21:23:00.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:00.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=33, tag_tid=5, entry_tid=11825' 2024-04-23T21:23:00.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=33, tag_tid=5, entry_tid=11825' 2024-04-23T21:23:00.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=33, tag_tid=5, entry_tid=11825' '!=' 'object_number=34, tag_tid=5, entry_tid=11610' 2024-04-23T21:23:00.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:00.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=33, tag_tid=5, entry_tid=11825' = 'object_number=34, tag_tid=5, entry_tid=11610' 2024-04-23T21:23:00.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=33, tag_tid=5, entry_tid=11825' 2024-04-23T21:23:00.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:00.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:00.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:00.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:00.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=33, tag_tid=5, entry_tid=11825' 2024-04-23T21:23:00.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:00.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:00.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:00.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:00.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:00.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:00.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:00.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=33, tag_tid=5, entry_tid=11825' 2024-04-23T21:23:00.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:00.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:00.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=11825 2024-04-23T21:23:00.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 11825 2024-04-23T21:23:00.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:00.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:00.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:00.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:00.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:00.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:00.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:00.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:00.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:00.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:00.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:00.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:00.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:00.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:00.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:00.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:00.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:00.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:00.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:00.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:00.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:00.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:00.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:00.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:00.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:00.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:00.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:00.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:00.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:00.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:00.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:00.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:00.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:00.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:00.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:00.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:00.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:00.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:00.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:00.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:00.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:00.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:00.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:00.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:00.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:00.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:00.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:00.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:00.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:00.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:00.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:00.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:00.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:00.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:00.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:00.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:00.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:00.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:00.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:00.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:00.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:00.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:00.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:00.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:00.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:00.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:00.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:00.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:00.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:00.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:00.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:00.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=35, tag_tid=5, entry_tid=12071], [object_number=34, tag_tid=5, entry_tid=12070], [object_number=33, tag_tid=5, entry_tid=12069], [object_number=32, tag_tid=5, entry_tid=12068]]], state=connected] 2024-04-23T21:23:00.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:00.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:00.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:00.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:00.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:00.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:00.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:00.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:00.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:00.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:00.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:00.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:00.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:00.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:00.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:00.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:00.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:00.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:00.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:00.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:00.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:00.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=35, tag_tid=5, entry_tid=12071], [object_number=34, tag_tid=5, entry_tid=12070], [object_number=33, tag_tid=5, entry_tid=12069], [object_number=32, tag_tid=5, entry_tid=12068]]], state=connected] 2024-04-23T21:23:00.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=35, tag_tid=5, entry_tid=12071' 2024-04-23T21:23:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=35, tag_tid=5, entry_tid=12071' 2024-04-23T21:23:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=35, tag_tid=5, entry_tid=12071' '!=' 'object_number=33, tag_tid=5, entry_tid=11825' 2024-04-23T21:23:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=35, tag_tid=5, entry_tid=12071' = 'object_number=33, tag_tid=5, entry_tid=11825' 2024-04-23T21:23:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=35, tag_tid=5, entry_tid=12071' 2024-04-23T21:23:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:00.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=35, tag_tid=5, entry_tid=12071' 2024-04-23T21:23:00.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:00.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:00.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:00.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:00.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:00.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:00.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:00.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=35, tag_tid=5, entry_tid=12071' 2024-04-23T21:23:00.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:00.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:00.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=12071 2024-04-23T21:23:00.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 12071 2024-04-23T21:23:00.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:00.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:00.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:01.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:01.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:01.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:01.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:01.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:01.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:01.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:01.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:01.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:01.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:01.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:01.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:01.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:01.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:01.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:01.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:01.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:01.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:01.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:01.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:01.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:01.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:01.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:01.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:01.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:01.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:01.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:01.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:01.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:01.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:01.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:01.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:01.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:01.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:01.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:01.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:01.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:01.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:01.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:01.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:01.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:01.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:01.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:01.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:01.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:01.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:01.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:01.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:01.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:01.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:01.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:01.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:01.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:01.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:01.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:01.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:01.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:01.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:01.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:01.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:01.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:01.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:01.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:01.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:01.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:01.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:01.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:01.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:01.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:01.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:01.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:01.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:01.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:01.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:01.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:01.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:01.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:01.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:01.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:01.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:01.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:01.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:01.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:01.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:01.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:01.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:01.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=35, tag_tid=5, entry_tid=12243], [object_number=34, tag_tid=5, entry_tid=12242], [object_number=33, tag_tid=5, entry_tid=12241], [object_number=32, tag_tid=5, entry_tid=12240]]], state=connected] 2024-04-23T21:23:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:01.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:01.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:01.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:01.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:01.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=35, tag_tid=5, entry_tid=12243], [object_number=34, tag_tid=5, entry_tid=12242], [object_number=33, tag_tid=5, entry_tid=12241], [object_number=32, tag_tid=5, entry_tid=12240]]], state=connected] 2024-04-23T21:23:01.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:01.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=35, tag_tid=5, entry_tid=12243' 2024-04-23T21:23:01.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=35, tag_tid=5, entry_tid=12243' 2024-04-23T21:23:01.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=35, tag_tid=5, entry_tid=12243' '!=' 'object_number=35, tag_tid=5, entry_tid=12071' 2024-04-23T21:23:01.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:01.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=35, tag_tid=5, entry_tid=12243' = 'object_number=35, tag_tid=5, entry_tid=12071' 2024-04-23T21:23:01.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=35, tag_tid=5, entry_tid=12243' 2024-04-23T21:23:01.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:01.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:01.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:01.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:01.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=35, tag_tid=5, entry_tid=12243' 2024-04-23T21:23:01.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:01.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:01.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:01.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:01.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:01.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:01.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:01.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=35, tag_tid=5, entry_tid=12243' 2024-04-23T21:23:01.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:01.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:01.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=12243 2024-04-23T21:23:01.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 12243 2024-04-23T21:23:01.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:01.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:01.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:01.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:01.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:01.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:01.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:01.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:01.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:01.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:01.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:01.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:01.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:01.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:01.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:01.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:01.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:01.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:01.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:01.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:01.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:01.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:01.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:01.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:01.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:01.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:01.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:01.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:01.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:01.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:01.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:01.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:01.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:01.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:01.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:01.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:01.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:01.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:01.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:01.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:01.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:01.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:01.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:01.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:01.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:01.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:01.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:01.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:01.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:01.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:01.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:01.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:01.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:01.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:01.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:01.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:01.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:01.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=35, tag_tid=5, entry_tid=12431], [object_number=34, tag_tid=5, entry_tid=12430], [object_number=33, tag_tid=5, entry_tid=12429], [object_number=32, tag_tid=5, entry_tid=12428]]], state=connected] 2024-04-23T21:23:01.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:01.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:01.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:01.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:01.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:01.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:01.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:01.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:01.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:01.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:01.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:01.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:01.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:01.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:01.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:01.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:01.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:01.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:01.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:01.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:01.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=35, tag_tid=5, entry_tid=12431], [object_number=34, tag_tid=5, entry_tid=12430], [object_number=33, tag_tid=5, entry_tid=12429], [object_number=32, tag_tid=5, entry_tid=12428]]], state=connected] 2024-04-23T21:23:01.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:01.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=35, tag_tid=5, entry_tid=12431' 2024-04-23T21:23:01.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=35, tag_tid=5, entry_tid=12431' 2024-04-23T21:23:01.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=35, tag_tid=5, entry_tid=12431' '!=' 'object_number=35, tag_tid=5, entry_tid=12243' 2024-04-23T21:23:01.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:01.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=35, tag_tid=5, entry_tid=12431' = 'object_number=35, tag_tid=5, entry_tid=12243' 2024-04-23T21:23:01.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=35, tag_tid=5, entry_tid=12431' 2024-04-23T21:23:01.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:01.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:01.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:01.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:01.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=35, tag_tid=5, entry_tid=12431' 2024-04-23T21:23:01.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:01.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:01.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:01.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:01.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:02.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:02.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:02.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=35, tag_tid=5, entry_tid=12431' 2024-04-23T21:23:02.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:02.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=12431 2024-04-23T21:23:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 12431 2024-04-23T21:23:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:02.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:02.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:02.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:02.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:02.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:02.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:02.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:02.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:02.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:02.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:02.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:02.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:02.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:02.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:02.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:02.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:02.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:02.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:02.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:02.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:02.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:02.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:02.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:02.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:02.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:02.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:02.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:02.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:02.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:02.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:02.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:02.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:02.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:02.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:02.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:02.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:02.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:02.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:02.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:02.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:02.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:02.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:02.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:02.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=34, tag_tid=5, entry_tid=12646], [object_number=33, tag_tid=5, entry_tid=12645], [object_number=32, tag_tid=5, entry_tid=12644], [object_number=35, tag_tid=5, entry_tid=12643]]], state=connected] 2024-04-23T21:23:02.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:02.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:02.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:02.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:02.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:02.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:02.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:02.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:02.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:02.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:02.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:02.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:02.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:02.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:02.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:02.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:02.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:02.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:02.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:02.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:02.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:02.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=34, tag_tid=5, entry_tid=12646], [object_number=33, tag_tid=5, entry_tid=12645], [object_number=32, tag_tid=5, entry_tid=12644], [object_number=35, tag_tid=5, entry_tid=12643]]], state=connected] 2024-04-23T21:23:02.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:02.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=34, tag_tid=5, entry_tid=12646' 2024-04-23T21:23:02.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=34, tag_tid=5, entry_tid=12646' 2024-04-23T21:23:02.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=34, tag_tid=5, entry_tid=12646' '!=' 'object_number=35, tag_tid=5, entry_tid=12431' 2024-04-23T21:23:02.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:02.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=34, tag_tid=5, entry_tid=12646' = 'object_number=35, tag_tid=5, entry_tid=12431' 2024-04-23T21:23:02.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=34, tag_tid=5, entry_tid=12646' 2024-04-23T21:23:02.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:02.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:02.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:02.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:02.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=34, tag_tid=5, entry_tid=12646' 2024-04-23T21:23:02.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:02.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:02.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:02.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:02.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:02.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:02.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:02.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=34, tag_tid=5, entry_tid=12646' 2024-04-23T21:23:02.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:02.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:02.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=12646 2024-04-23T21:23:02.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 12646 2024-04-23T21:23:02.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:02.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:02.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:02.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:02.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:02.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:02.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:02.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:02.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:02.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:02.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:02.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:02.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:02.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:02.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:02.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:02.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:02.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:02.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:02.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:02.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:02.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:02.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:02.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:02.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:02.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:02.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:02.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:02.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:02.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:02.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:02.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:02.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:02.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:02.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:02.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:02.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:02.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:02.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:02.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:02.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:02.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:02.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:02.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:02.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:02.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:02.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:02.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:02.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:02.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:02.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:02.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:02.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:02.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:02.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:02.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:02.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:02.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:02.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:02.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:02.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:02.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:02.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:02.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:02.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:02.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:02.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=33, tag_tid=5, entry_tid=12849], [object_number=32, tag_tid=5, entry_tid=12848], [object_number=35, tag_tid=5, entry_tid=12847], [object_number=34, tag_tid=5, entry_tid=12846]]], state=connected] 2024-04-23T21:23:02.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:02.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:02.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:02.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:02.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:02.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:02.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:02.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:02.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:02.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:02.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:02.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:02.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:02.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:02.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:02.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:02.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:03.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:03.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:03.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:03.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:03.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=33, tag_tid=5, entry_tid=12849], [object_number=32, tag_tid=5, entry_tid=12848], [object_number=35, tag_tid=5, entry_tid=12847], [object_number=34, tag_tid=5, entry_tid=12846]]], state=connected] 2024-04-23T21:23:03.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:03.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=33, tag_tid=5, entry_tid=12849' 2024-04-23T21:23:03.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=33, tag_tid=5, entry_tid=12849' 2024-04-23T21:23:03.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=33, tag_tid=5, entry_tid=12849' '!=' 'object_number=34, tag_tid=5, entry_tid=12646' 2024-04-23T21:23:03.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:03.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=33, tag_tid=5, entry_tid=12849' = 'object_number=34, tag_tid=5, entry_tid=12646' 2024-04-23T21:23:03.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=33, tag_tid=5, entry_tid=12849' 2024-04-23T21:23:03.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:03.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:03.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:03.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:03.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=33, tag_tid=5, entry_tid=12849' 2024-04-23T21:23:03.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:03.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:03.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:03.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:03.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:03.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:03.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:03.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=33, tag_tid=5, entry_tid=12849' 2024-04-23T21:23:03.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:03.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:03.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=12849 2024-04-23T21:23:03.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 12849 2024-04-23T21:23:03.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:03.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:03.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:03.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:03.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:03.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:03.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:03.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:03.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:03.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:03.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:03.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:03.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:03.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:03.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:03.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:03.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:03.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:03.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:03.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:03.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:03.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:03.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:03.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:03.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:03.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:03.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:03.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:03.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:03.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:03.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:03.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:03.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:03.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:03.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:03.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:03.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:03.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:03.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:03.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:03.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:03.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:03.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:03.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:03.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:03.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:03.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:03.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:03.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:03.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:03.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:03.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:03.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:03.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:03.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:03.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:03.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:03.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:03.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:03.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:03.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:03.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:03.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:03.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:03.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:03.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:03.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:03.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:03.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:03.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:03.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:03.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:03.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:03.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:03.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:03.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:03.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:03.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:03.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:03.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:03.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:03.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:03.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:03.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:03.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:03.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:03.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:03.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:03.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:03.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:03.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:03.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=33, tag_tid=5, entry_tid=13033], [object_number=32, tag_tid=5, entry_tid=13032], [object_number=35, tag_tid=5, entry_tid=13031], [object_number=34, tag_tid=5, entry_tid=13030]]], state=connected] 2024-04-23T21:23:03.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:03.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:03.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:03.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:03.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:03.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:03.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:03.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:03.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:03.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:03.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:03.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:03.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:03.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:03.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:03.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:03.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:03.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:03.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:03.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:03.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:03.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=33, tag_tid=5, entry_tid=13033], [object_number=32, tag_tid=5, entry_tid=13032], [object_number=35, tag_tid=5, entry_tid=13031], [object_number=34, tag_tid=5, entry_tid=13030]]], state=connected] 2024-04-23T21:23:03.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:03.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=33, tag_tid=5, entry_tid=13033' 2024-04-23T21:23:03.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=33, tag_tid=5, entry_tid=13033' 2024-04-23T21:23:03.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=33, tag_tid=5, entry_tid=13033' '!=' 'object_number=33, tag_tid=5, entry_tid=12849' 2024-04-23T21:23:03.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:03.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=33, tag_tid=5, entry_tid=13033' = 'object_number=33, tag_tid=5, entry_tid=12849' 2024-04-23T21:23:03.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=33, tag_tid=5, entry_tid=13033' 2024-04-23T21:23:03.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:03.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:03.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:03.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:03.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=33, tag_tid=5, entry_tid=13033' 2024-04-23T21:23:03.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:03.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:03.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:03.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:03.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:03.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:03.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:03.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=33, tag_tid=5, entry_tid=13033' 2024-04-23T21:23:03.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:03.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:03.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=13033 2024-04-23T21:23:03.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 13033 2024-04-23T21:23:03.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:03.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:03.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:03.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:03.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:03.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:03.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:03.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:03.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:03.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:03.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:03.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:03.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:03.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:03.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:03.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:03.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:03.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:03.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:03.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:03.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:03.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:03.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:03.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:03.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:03.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:03.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:03.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:03.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:03.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:03.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:03.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:03.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:03.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:03.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:03.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:03.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:03.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:03.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:03.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:03.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:03.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:03.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:03.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:03.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:03.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:03.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:03.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:03.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:03.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:03.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:03.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:03.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:03.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:03.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:03.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:03.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:03.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:03.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:03.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:03.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:03.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:03.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:03.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:03.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:03.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:03.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:03.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:03.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:03.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:03.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:03.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:03.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:03.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:03.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:03.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:03.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:03.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:03.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:03.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:03.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:03.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:03.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:03.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:03.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:03.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:03.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:03.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:03.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:03.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:03.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:03.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:03.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:03.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:03.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:03.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:03.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:03.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:03.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:03.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:03.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:03.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:03.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=35, tag_tid=5, entry_tid=13231], [object_number=34, tag_tid=5, entry_tid=13230], [object_number=33, tag_tid=5, entry_tid=13229], [object_number=32, tag_tid=5, entry_tid=13228]]], state=connected] 2024-04-23T21:23:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:03.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:03.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:03.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:03.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:03.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:03.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:03.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:03.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:03.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:04.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:04.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:04.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:04.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:04.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=35, tag_tid=5, entry_tid=13231], [object_number=34, tag_tid=5, entry_tid=13230], [object_number=33, tag_tid=5, entry_tid=13229], [object_number=32, tag_tid=5, entry_tid=13228]]], state=connected] 2024-04-23T21:23:04.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:04.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=35, tag_tid=5, entry_tid=13231' 2024-04-23T21:23:04.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=35, tag_tid=5, entry_tid=13231' 2024-04-23T21:23:04.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=35, tag_tid=5, entry_tid=13231' '!=' 'object_number=33, tag_tid=5, entry_tid=13033' 2024-04-23T21:23:04.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:04.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=35, tag_tid=5, entry_tid=13231' = 'object_number=33, tag_tid=5, entry_tid=13033' 2024-04-23T21:23:04.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=35, tag_tid=5, entry_tid=13231' 2024-04-23T21:23:04.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:04.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:04.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:04.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:04.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=35, tag_tid=5, entry_tid=13231' 2024-04-23T21:23:04.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:04.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:04.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:04.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:04.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:04.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:04.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:04.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=35, tag_tid=5, entry_tid=13231' 2024-04-23T21:23:04.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:04.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:04.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=13231 2024-04-23T21:23:04.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 13231 2024-04-23T21:23:04.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:04.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:04.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:04.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:04.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:04.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:04.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:04.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:04.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:04.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:04.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:04.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:04.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:04.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:04.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:04.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:04.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:04.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:04.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:04.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:04.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:04.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:04.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:04.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:04.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:04.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:04.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:04.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:04.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:04.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:04.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:04.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:04.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:04.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:04.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:04.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:04.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:04.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:04.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:04.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:04.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:04.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:04.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:04.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:04.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:04.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:04.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:04.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:04.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:04.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:04.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:04.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:04.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:04.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:04.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:04.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:04.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:04.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:04.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:04.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:04.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:04.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:04.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:04.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:04.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:04.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:04.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:04.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:04.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:04.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:04.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:04.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:04.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:04.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:04.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:04.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:04.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:04.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:04.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:04.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:04.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:04.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:04.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:04.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:04.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:04.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:04.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:04.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:04.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:04.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:04.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:04.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:04.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:04.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:04.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:04.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:04.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:04.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=32, tag_tid=5, entry_tid=13436], [object_number=35, tag_tid=5, entry_tid=13435], [object_number=34, tag_tid=5, entry_tid=13434], [object_number=33, tag_tid=5, entry_tid=13433]]], state=connected] 2024-04-23T21:23:04.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:04.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:04.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:04.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:04.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:04.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:04.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:04.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:04.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:04.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:04.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:04.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:04.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:04.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:04.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:04.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:04.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:04.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:04.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:04.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:04.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:04.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=32, tag_tid=5, entry_tid=13436], [object_number=35, tag_tid=5, entry_tid=13435], [object_number=34, tag_tid=5, entry_tid=13434], [object_number=33, tag_tid=5, entry_tid=13433]]], state=connected] 2024-04-23T21:23:04.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:04.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=32, tag_tid=5, entry_tid=13436' 2024-04-23T21:23:04.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=32, tag_tid=5, entry_tid=13436' 2024-04-23T21:23:04.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=32, tag_tid=5, entry_tid=13436' '!=' 'object_number=35, tag_tid=5, entry_tid=13231' 2024-04-23T21:23:04.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:04.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=32, tag_tid=5, entry_tid=13436' = 'object_number=35, tag_tid=5, entry_tid=13231' 2024-04-23T21:23:04.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=32, tag_tid=5, entry_tid=13436' 2024-04-23T21:23:04.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:04.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:04.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:04.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:04.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=32, tag_tid=5, entry_tid=13436' 2024-04-23T21:23:04.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:04.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:04.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:04.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:04.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:04.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:04.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:04.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=32, tag_tid=5, entry_tid=13436' 2024-04-23T21:23:04.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:04.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:04.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=13436 2024-04-23T21:23:04.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 13436 2024-04-23T21:23:04.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:04.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:04.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:04.715 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2024-04-23T21:23:04.715 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2024-04-23T21:23:04.716 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2024-04-23T21:23:04.716 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:23:04.717+0000 7fe7a64b7640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 37228) UID: 1000 2024-04-23T21:23:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:04.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:04.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:04.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:04.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:04.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:04.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:04.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:04.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:04.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:04.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:04.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:04.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:04.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:04.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:04.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:04.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:04.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:04.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:04.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:04.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:04.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:04.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:04.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:04.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:04.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:04.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:04.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:04.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:04.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:04.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:04.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:04.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:04.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:04.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:04.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:04.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:04.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:04.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:04.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:04.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:04.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:05.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:05.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:05.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:05.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:05.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=33, tag_tid=5, entry_tid=13641], [object_number=32, tag_tid=5, entry_tid=13640], [object_number=35, tag_tid=5, entry_tid=13639], [object_number=34, tag_tid=5, entry_tid=13638]]], state=connected] 2024-04-23T21:23:05.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:05.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:05.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:05.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:05.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:05.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:05.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:05.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:05.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:05.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:05.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:05.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:05.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:05.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:05.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:05.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:05.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:05.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:05.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:05.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:05.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:05.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=33, tag_tid=5, entry_tid=13641], [object_number=32, tag_tid=5, entry_tid=13640], [object_number=35, tag_tid=5, entry_tid=13639], [object_number=34, tag_tid=5, entry_tid=13638]]], state=connected] 2024-04-23T21:23:05.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:05.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=33, tag_tid=5, entry_tid=13641' 2024-04-23T21:23:05.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=33, tag_tid=5, entry_tid=13641' 2024-04-23T21:23:05.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=33, tag_tid=5, entry_tid=13641' '!=' 'object_number=32, tag_tid=5, entry_tid=13436' 2024-04-23T21:23:05.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:05.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=33, tag_tid=5, entry_tid=13641' = 'object_number=32, tag_tid=5, entry_tid=13436' 2024-04-23T21:23:05.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=33, tag_tid=5, entry_tid=13641' 2024-04-23T21:23:05.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:05.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:05.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:05.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:05.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=33, tag_tid=5, entry_tid=13641' 2024-04-23T21:23:05.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:05.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:05.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=33, tag_tid=5, entry_tid=13641' 2024-04-23T21:23:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:05.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=13641 2024-04-23T21:23:05.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 13641 2024-04-23T21:23:05.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:05.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:05.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:05.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:05.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:05.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:05.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:05.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:05.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:05.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:05.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:05.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:05.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:05.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:05.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:05.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:05.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:05.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:05.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:05.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:05.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:05.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:05.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:05.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:05.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:05.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:05.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:05.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:05.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:05.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:05.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:05.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:05.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:05.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:05.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:05.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:05.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:05.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:05.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:05.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:05.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:05.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:05.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:05.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:05.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:05.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:05.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:05.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:05.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:05.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:05.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:05.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:05.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:05.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:05.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:05.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:05.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:05.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:05.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:05.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:05.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:05.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:05.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:05.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:05.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:05.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:05.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:05.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:05.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:05.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:05.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:05.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:05.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:05.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:05.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:05.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:05.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:05.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:05.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:05.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:05.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:05.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:05.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:05.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:05.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:05.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:05.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:05.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=34, tag_tid=5, entry_tid=13822], [object_number=33, tag_tid=5, entry_tid=13821], [object_number=32, tag_tid=5, entry_tid=13820], [object_number=35, tag_tid=5, entry_tid=13819]]], state=connected] 2024-04-23T21:23:05.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:05.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:05.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:05.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:05.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:05.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:05.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:05.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:05.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:05.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:05.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:05.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:05.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:05.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:05.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:05.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:05.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:05.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:05.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:05.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:05.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:05.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=34, tag_tid=5, entry_tid=13822], [object_number=33, tag_tid=5, entry_tid=13821], [object_number=32, tag_tid=5, entry_tid=13820], [object_number=35, tag_tid=5, entry_tid=13819]]], state=connected] 2024-04-23T21:23:05.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:05.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=34, tag_tid=5, entry_tid=13822' 2024-04-23T21:23:05.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=34, tag_tid=5, entry_tid=13822' 2024-04-23T21:23:05.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=34, tag_tid=5, entry_tid=13822' '!=' 'object_number=33, tag_tid=5, entry_tid=13641' 2024-04-23T21:23:05.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:05.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=34, tag_tid=5, entry_tid=13822' = 'object_number=33, tag_tid=5, entry_tid=13641' 2024-04-23T21:23:05.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=34, tag_tid=5, entry_tid=13822' 2024-04-23T21:23:05.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:05.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:05.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=34, tag_tid=5, entry_tid=13822' 2024-04-23T21:23:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:05.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:05.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:05.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:05.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:05.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:05.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:05.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=34, tag_tid=5, entry_tid=13822' 2024-04-23T21:23:05.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:05.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=13822 2024-04-23T21:23:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 13822 2024-04-23T21:23:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:05.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:05.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:05.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:05.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:05.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:05.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:05.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:05.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:05.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:05.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:05.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:05.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:05.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:05.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:05.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:05.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:05.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:05.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:05.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:05.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:05.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:05.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:05.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:05.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:05.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:05.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:05.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:05.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:05.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:05.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:05.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:05.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:05.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:05.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:05.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:05.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:05.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:05.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:05.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:05.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:05.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:05.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:05.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:05.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:05.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:05.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:05.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:05.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:05.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:05.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:05.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:05.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:05.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:05.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:05.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:05.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:05.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:05.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:05.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:06.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:06.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:06.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:06.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:06.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=33, tag_tid=5, entry_tid=14017], [object_number=32, tag_tid=5, entry_tid=14016], [object_number=35, tag_tid=5, entry_tid=14015], [object_number=34, tag_tid=5, entry_tid=14014]]], state=connected] 2024-04-23T21:23:06.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:06.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:06.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:06.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:06.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:06.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:06.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:06.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:06.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:06.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:06.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:06.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:06.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:06.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:06.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:06.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:06.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:06.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:06.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:06.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:06.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:06.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=33, tag_tid=5, entry_tid=14017], [object_number=32, tag_tid=5, entry_tid=14016], [object_number=35, tag_tid=5, entry_tid=14015], [object_number=34, tag_tid=5, entry_tid=14014]]], state=connected] 2024-04-23T21:23:06.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:06.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=33, tag_tid=5, entry_tid=14017' 2024-04-23T21:23:06.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=33, tag_tid=5, entry_tid=14017' 2024-04-23T21:23:06.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=33, tag_tid=5, entry_tid=14017' '!=' 'object_number=34, tag_tid=5, entry_tid=13822' 2024-04-23T21:23:06.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:06.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=33, tag_tid=5, entry_tid=14017' = 'object_number=34, tag_tid=5, entry_tid=13822' 2024-04-23T21:23:06.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=33, tag_tid=5, entry_tid=14017' 2024-04-23T21:23:06.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:06.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:06.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:06.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:06.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=33, tag_tid=5, entry_tid=14017' 2024-04-23T21:23:06.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:06.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:06.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:06.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:06.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:06.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:06.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:06.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=33, tag_tid=5, entry_tid=14017' 2024-04-23T21:23:06.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:06.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:06.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=14017 2024-04-23T21:23:06.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 14017 2024-04-23T21:23:06.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:06.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:06.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:06.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:06.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:06.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:06.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:06.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:06.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:06.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:06.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:06.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:06.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:06.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:06.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:06.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:06.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:06.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:06.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:06.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:06.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:06.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:06.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:06.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:06.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:06.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:06.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:06.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:06.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:06.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:06.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:06.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:06.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:06.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:06.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:06.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:06.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:06.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:06.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:06.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:06.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:06.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:06.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:06.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:06.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:06.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:06.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:06.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:06.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:06.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:06.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:06.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:06.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:06.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:06.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:06.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:06.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:06.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:06.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:06.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:06.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:06.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:06.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:06.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:06.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:06.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:06.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:06.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:06.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:06.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:06.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:06.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:06.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:06.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:06.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:06.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:06.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:06.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:06.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:06.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=35, tag_tid=5, entry_tid=14211], [object_number=34, tag_tid=5, entry_tid=14210], [object_number=33, tag_tid=5, entry_tid=14209], [object_number=32, tag_tid=5, entry_tid=14208]]], state=connected] 2024-04-23T21:23:06.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:06.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:06.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:06.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:06.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:06.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:06.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:06.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:06.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:06.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:06.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:06.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:06.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:06.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:06.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:06.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:06.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:06.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:06.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:06.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:06.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:06.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=35, tag_tid=5, entry_tid=14211], [object_number=34, tag_tid=5, entry_tid=14210], [object_number=33, tag_tid=5, entry_tid=14209], [object_number=32, tag_tid=5, entry_tid=14208]]], state=connected] 2024-04-23T21:23:06.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=35, tag_tid=5, entry_tid=14211' 2024-04-23T21:23:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=35, tag_tid=5, entry_tid=14211' 2024-04-23T21:23:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=35, tag_tid=5, entry_tid=14211' '!=' 'object_number=33, tag_tid=5, entry_tid=14017' 2024-04-23T21:23:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=35, tag_tid=5, entry_tid=14211' = 'object_number=33, tag_tid=5, entry_tid=14017' 2024-04-23T21:23:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=35, tag_tid=5, entry_tid=14211' 2024-04-23T21:23:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:06.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:06.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=35, tag_tid=5, entry_tid=14211' 2024-04-23T21:23:06.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:06.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:06.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:06.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:06.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:06.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:06.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=35, tag_tid=5, entry_tid=14211' 2024-04-23T21:23:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=14211 2024-04-23T21:23:06.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 14211 2024-04-23T21:23:06.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:06.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:06.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:06.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:06.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:06.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:06.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:06.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:06.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:06.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:06.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:06.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:06.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:06.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:06.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:06.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:06.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:06.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:06.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:06.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:06.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:06.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:06.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:06.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:06.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:06.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:06.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:06.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:06.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:06.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:06.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:06.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:06.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:06.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:06.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:06.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:06.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:06.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:06.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:06.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:06.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:06.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:06.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:06.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:06.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:06.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:06.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:06.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:06.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:06.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:06.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:06.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:06.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:06.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:06.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:06.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:06.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:06.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:06.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:06.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:06.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:06.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:06.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:06.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:06.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:06.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:06.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:06.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:06.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:06.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:06.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:07.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:07.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:07.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:07.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:07.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:07.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:07.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:07.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:07.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=34, tag_tid=5, entry_tid=14394], [object_number=33, tag_tid=5, entry_tid=14393], [object_number=32, tag_tid=5, entry_tid=14392], [object_number=35, tag_tid=5, entry_tid=14391]]], state=connected] 2024-04-23T21:23:07.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:07.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:07.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:07.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:07.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:07.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:07.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:07.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:07.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:07.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:07.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:07.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:07.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:07.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:07.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:07.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:07.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:07.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:07.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:07.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:07.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:07.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=34, tag_tid=5, entry_tid=14394], [object_number=33, tag_tid=5, entry_tid=14393], [object_number=32, tag_tid=5, entry_tid=14392], [object_number=35, tag_tid=5, entry_tid=14391]]], state=connected] 2024-04-23T21:23:07.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:07.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=34, tag_tid=5, entry_tid=14394' 2024-04-23T21:23:07.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=34, tag_tid=5, entry_tid=14394' 2024-04-23T21:23:07.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=34, tag_tid=5, entry_tid=14394' '!=' 'object_number=35, tag_tid=5, entry_tid=14211' 2024-04-23T21:23:07.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:07.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=34, tag_tid=5, entry_tid=14394' = 'object_number=35, tag_tid=5, entry_tid=14211' 2024-04-23T21:23:07.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=34, tag_tid=5, entry_tid=14394' 2024-04-23T21:23:07.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:07.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:07.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:07.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:07.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=34, tag_tid=5, entry_tid=14394' 2024-04-23T21:23:07.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:07.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:07.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:07.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:07.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:07.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:07.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:07.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=34, tag_tid=5, entry_tid=14394' 2024-04-23T21:23:07.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:07.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:07.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=14394 2024-04-23T21:23:07.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 14394 2024-04-23T21:23:07.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:07.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:07.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:07.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:07.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:07.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:07.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:07.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:07.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:07.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:07.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:07.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:07.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:07.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:07.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:07.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:07.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:07.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:07.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:07.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:07.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:07.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:07.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:07.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:07.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:07.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:07.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:07.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:07.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:07.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:07.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:07.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:07.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:07.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:07.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:07.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:07.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:07.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:07.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:07.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:07.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:07.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:07.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:07.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:07.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:07.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:07.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:07.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:07.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:07.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:07.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:07.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:07.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:07.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:07.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:07.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:07.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:07.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:07.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:07.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:07.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:07.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=34, tag_tid=5, entry_tid=14594], [object_number=33, tag_tid=5, entry_tid=14593], [object_number=32, tag_tid=5, entry_tid=14592], [object_number=35, tag_tid=5, entry_tid=14591]]], state=connected] 2024-04-23T21:23:07.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:07.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:07.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:07.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:07.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:07.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:07.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:07.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:07.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:07.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:07.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:07.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:07.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:07.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:07.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:07.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:07.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:07.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:07.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:07.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:07.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:07.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=34, tag_tid=5, entry_tid=14594], [object_number=33, tag_tid=5, entry_tid=14593], [object_number=32, tag_tid=5, entry_tid=14592], [object_number=35, tag_tid=5, entry_tid=14591]]], state=connected] 2024-04-23T21:23:07.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:07.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=34, tag_tid=5, entry_tid=14594' 2024-04-23T21:23:07.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=34, tag_tid=5, entry_tid=14594' 2024-04-23T21:23:07.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=34, tag_tid=5, entry_tid=14594' '!=' 'object_number=34, tag_tid=5, entry_tid=14394' 2024-04-23T21:23:07.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:07.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=34, tag_tid=5, entry_tid=14594' = 'object_number=34, tag_tid=5, entry_tid=14394' 2024-04-23T21:23:07.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=34, tag_tid=5, entry_tid=14594' 2024-04-23T21:23:07.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:07.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:07.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:07.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:07.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=34, tag_tid=5, entry_tid=14594' 2024-04-23T21:23:07.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:07.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:07.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=34, tag_tid=5, entry_tid=14594' 2024-04-23T21:23:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:07.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:07.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=14594 2024-04-23T21:23:07.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 14594 2024-04-23T21:23:07.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:07.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:07.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:07.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:07.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:07.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:07.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:07.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:07.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:07.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:07.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:07.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:07.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:07.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:07.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:07.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:07.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:07.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:07.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:07.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:07.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:07.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:07.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:07.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:07.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:07.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:07.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:07.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:07.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:07.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:07.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:07.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:07.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:07.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:07.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:07.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:07.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:07.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:07.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:07.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:07.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:07.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:07.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:07.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:07.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:07.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:07.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:07.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:07.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:07.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:07.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:07.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:07.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:07.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:07.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:07.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:07.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:07.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:08.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:08.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:08.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:08.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:08.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:08.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:08.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:08.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:08.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:08.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:08.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:08.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:08.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:08.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:08.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:08.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:08.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:08.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:08.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:08.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:08.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=33, tag_tid=5, entry_tid=14785], [object_number=32, tag_tid=5, entry_tid=14784], [object_number=35, tag_tid=5, entry_tid=14783], [object_number=34, tag_tid=5, entry_tid=14782]]], state=connected] 2024-04-23T21:23:08.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:08.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:08.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:08.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:08.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:08.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:08.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:08.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:08.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:08.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:08.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:08.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:08.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:08.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:08.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:08.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:08.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:08.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:08.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:08.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:08.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:08.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=33, tag_tid=5, entry_tid=14785], [object_number=32, tag_tid=5, entry_tid=14784], [object_number=35, tag_tid=5, entry_tid=14783], [object_number=34, tag_tid=5, entry_tid=14782]]], state=connected] 2024-04-23T21:23:08.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:08.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=33, tag_tid=5, entry_tid=14785' 2024-04-23T21:23:08.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=33, tag_tid=5, entry_tid=14785' 2024-04-23T21:23:08.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=33, tag_tid=5, entry_tid=14785' '!=' 'object_number=34, tag_tid=5, entry_tid=14594' 2024-04-23T21:23:08.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:08.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=33, tag_tid=5, entry_tid=14785' = 'object_number=34, tag_tid=5, entry_tid=14594' 2024-04-23T21:23:08.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=33, tag_tid=5, entry_tid=14785' 2024-04-23T21:23:08.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:08.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:08.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:08.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:08.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=33, tag_tid=5, entry_tid=14785' 2024-04-23T21:23:08.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:08.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:08.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:08.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:08.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:08.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:08.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:08.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=33, tag_tid=5, entry_tid=14785' 2024-04-23T21:23:08.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:08.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:08.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=14785 2024-04-23T21:23:08.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 14785 2024-04-23T21:23:08.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:08.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:08.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:08.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:08.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:08.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:08.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:08.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:08.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:08.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:08.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:08.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:08.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:08.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:08.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:08.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:08.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:08.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:08.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:08.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:08.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:08.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:08.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:08.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:08.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:08.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:08.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:08.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:08.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:08.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:08.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:08.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:08.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:08.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:08.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:08.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:08.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:08.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:08.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:08.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:08.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:08.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:08.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:08.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:08.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:08.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:08.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:08.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:08.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:08.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:08.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:08.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:08.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:08.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:08.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:08.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:08.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:08.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:08.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:08.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:08.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:08.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:08.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:08.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:08.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:08.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:08.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:08.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:08.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:08.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:08.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:08.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:08.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:08.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:08.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:08.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:08.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:08.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:08.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:08.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:08.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:08.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:08.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:08.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:08.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:08.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:08.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:08.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:08.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:08.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:08.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:08.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:08.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:08.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:08.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:08.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:08.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:08.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:08.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:08.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:08.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:08.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:08.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:08.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:08.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:08.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:08.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:08.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:08.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:08.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:08.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:08.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:08.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:08.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:08.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:08.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=35, tag_tid=5, entry_tid=14979], [object_number=34, tag_tid=5, entry_tid=14978], [object_number=33, tag_tid=5, entry_tid=14977], [object_number=32, tag_tid=5, entry_tid=14976]]], state=connected] 2024-04-23T21:23:08.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:08.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:08.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:08.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:08.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:08.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:08.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:08.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:08.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:08.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:08.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:08.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:08.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:08.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:08.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:08.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:08.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:08.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:08.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:08.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:08.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:08.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=35, tag_tid=5, entry_tid=14979], [object_number=34, tag_tid=5, entry_tid=14978], [object_number=33, tag_tid=5, entry_tid=14977], [object_number=32, tag_tid=5, entry_tid=14976]]], state=connected] 2024-04-23T21:23:08.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=35, tag_tid=5, entry_tid=14979' 2024-04-23T21:23:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=35, tag_tid=5, entry_tid=14979' 2024-04-23T21:23:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=35, tag_tid=5, entry_tid=14979' '!=' 'object_number=33, tag_tid=5, entry_tid=14785' 2024-04-23T21:23:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=35, tag_tid=5, entry_tid=14979' = 'object_number=33, tag_tid=5, entry_tid=14785' 2024-04-23T21:23:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=35, tag_tid=5, entry_tid=14979' 2024-04-23T21:23:08.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=35, tag_tid=5, entry_tid=14979' 2024-04-23T21:23:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:08.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:08.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:08.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:08.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:08.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:08.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=35, tag_tid=5, entry_tid=14979' 2024-04-23T21:23:08.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:08.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=14979 2024-04-23T21:23:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 14979 2024-04-23T21:23:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:08.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:08.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:08.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:08.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:08.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:08.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:08.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:08.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:08.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:08.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:08.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:08.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:08.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:08.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:08.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:08.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:08.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:08.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:08.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:08.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:08.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:08.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:08.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:08.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:08.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:08.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:08.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:08.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:08.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:08.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:08.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:08.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:08.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:08.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:08.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:08.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:08.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:08.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:08.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:08.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:08.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:08.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:08.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:08.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:08.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:08.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:08.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:08.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:08.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:08.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:08.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:08.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:08.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:08.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:08.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:08.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:08.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:08.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:08.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:08.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:08.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:08.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:08.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:08.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:08.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:08.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:08.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:08.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:08.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:08.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:08.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:08.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:08.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:08.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:08.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:08.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:08.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:08.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:08.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:09.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:09.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:09.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:09.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:09.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:09.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:09.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:09.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:09.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:09.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:09.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:09.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:09.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:09.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:09.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:09.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:09.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:09.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:09.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=32, tag_tid=5, entry_tid=15184], [object_number=35, tag_tid=5, entry_tid=15183], [object_number=34, tag_tid=5, entry_tid=15182], [object_number=33, tag_tid=5, entry_tid=15181]]], state=connected] 2024-04-23T21:23:09.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:09.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:09.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:09.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:09.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:09.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:09.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:09.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:09.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:09.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:09.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:09.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:09.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:09.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:09.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:09.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:09.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:09.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:09.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:09.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:09.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:09.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=32, tag_tid=5, entry_tid=15184], [object_number=35, tag_tid=5, entry_tid=15183], [object_number=34, tag_tid=5, entry_tid=15182], [object_number=33, tag_tid=5, entry_tid=15181]]], state=connected] 2024-04-23T21:23:09.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:09.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=32, tag_tid=5, entry_tid=15184' 2024-04-23T21:23:09.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=32, tag_tid=5, entry_tid=15184' 2024-04-23T21:23:09.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=32, tag_tid=5, entry_tid=15184' '!=' 'object_number=35, tag_tid=5, entry_tid=14979' 2024-04-23T21:23:09.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:09.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=32, tag_tid=5, entry_tid=15184' = 'object_number=35, tag_tid=5, entry_tid=14979' 2024-04-23T21:23:09.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=32, tag_tid=5, entry_tid=15184' 2024-04-23T21:23:09.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:09.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:09.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:09.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:09.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=32, tag_tid=5, entry_tid=15184' 2024-04-23T21:23:09.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:09.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:09.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:09.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:09.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:09.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:09.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:09.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=32, tag_tid=5, entry_tid=15184' 2024-04-23T21:23:09.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:09.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:09.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=15184 2024-04-23T21:23:09.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 15184 2024-04-23T21:23:09.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:09.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:09.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:09.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:09.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:09.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:09.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:09.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:09.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:09.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:09.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:09.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:09.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:09.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:09.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:09.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:09.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:09.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:09.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:09.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:09.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:09.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:09.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:09.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:09.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:09.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:09.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:09.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:09.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:09.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:09.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:09.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:09.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:09.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:09.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:09.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:09.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:09.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:09.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:09.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:09.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:09.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:09.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:09.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:09.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:09.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:09.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:09.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:09.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:09.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:09.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:09.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:09.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:09.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:09.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:09.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:09.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:09.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:09.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:09.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:09.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:09.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:09.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:09.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:09.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:09.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:09.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:09.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:09.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:09.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:09.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:09.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:09.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:09.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:09.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:09.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:09.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:09.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:09.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:09.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:09.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:09.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:09.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:09.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:09.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:09.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:09.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:09.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:09.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:09.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:09.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:09.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:09.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:09.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:09.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=33, tag_tid=5, entry_tid=15377], [object_number=32, tag_tid=5, entry_tid=15376], [object_number=35, tag_tid=5, entry_tid=15375], [object_number=34, tag_tid=5, entry_tid=15374]]], state=connected] 2024-04-23T21:23:09.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:09.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:09.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:09.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:09.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:09.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:09.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:09.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:09.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:09.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:09.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:09.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:09.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:09.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:09.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:09.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:09.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:09.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:09.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:09.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:09.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:09.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=33, tag_tid=5, entry_tid=15377], [object_number=32, tag_tid=5, entry_tid=15376], [object_number=35, tag_tid=5, entry_tid=15375], [object_number=34, tag_tid=5, entry_tid=15374]]], state=connected] 2024-04-23T21:23:09.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:09.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=33, tag_tid=5, entry_tid=15377' 2024-04-23T21:23:09.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=33, tag_tid=5, entry_tid=15377' 2024-04-23T21:23:09.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=33, tag_tid=5, entry_tid=15377' '!=' 'object_number=32, tag_tid=5, entry_tid=15184' 2024-04-23T21:23:09.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:09.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=33, tag_tid=5, entry_tid=15377' = 'object_number=32, tag_tid=5, entry_tid=15184' 2024-04-23T21:23:09.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=33, tag_tid=5, entry_tid=15377' 2024-04-23T21:23:09.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:09.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:09.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:09.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:09.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=33, tag_tid=5, entry_tid=15377' 2024-04-23T21:23:09.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:09.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:09.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:09.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:09.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:09.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:09.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:09.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=33, tag_tid=5, entry_tid=15377' 2024-04-23T21:23:09.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:09.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:09.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=15377 2024-04-23T21:23:09.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 15377 2024-04-23T21:23:09.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:09.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:09.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:09.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:09.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:09.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:09.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:09.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:09.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:09.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:09.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:09.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:09.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:09.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:09.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:09.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:09.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:09.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:09.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:09.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:09.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:09.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:09.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:09.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:09.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:09.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:09.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:09.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:09.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:09.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:09.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:09.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:09.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:09.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:09.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:09.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:09.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:09.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:09.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:09.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:09.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:09.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:09.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:09.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:09.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:09.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:09.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:09.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:09.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:09.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:09.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:09.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:09.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:09.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:09.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:09.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:09.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:09.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:09.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:09.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:09.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:09.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:09.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:09.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:09.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:09.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:09.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:09.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:09.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:10.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:10.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:10.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:10.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:10.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:10.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:10.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:10.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:10.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:10.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:10.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:10.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:10.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:10.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:10.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:10.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:10.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:10.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:10.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:10.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:10.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=39, tag_tid=5, entry_tid=15527], [object_number=38, tag_tid=5, entry_tid=15526], [object_number=37, tag_tid=5, entry_tid=15525], [object_number=32, tag_tid=5, entry_tid=15524]]], state=connected] 2024-04-23T21:23:10.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:10.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:10.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:10.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:10.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:10.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:10.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:10.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:10.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:10.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:10.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:10.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:10.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:10.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:10.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:10.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:10.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:10.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 8 2024-04-23T21:23:10.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:10.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:10.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:10.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=39, tag_tid=5, entry_tid=15527], [object_number=38, tag_tid=5, entry_tid=15526], [object_number=37, tag_tid=5, entry_tid=15525], [object_number=32, tag_tid=5, entry_tid=15524]]], state=connected] 2024-04-23T21:23:10.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:10.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=39, tag_tid=5, entry_tid=15527' 2024-04-23T21:23:10.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=39, tag_tid=5, entry_tid=15527' 2024-04-23T21:23:10.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=39, tag_tid=5, entry_tid=15527' '!=' 'object_number=33, tag_tid=5, entry_tid=15377' 2024-04-23T21:23:10.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:10.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=39, tag_tid=5, entry_tid=15527' = 'object_number=33, tag_tid=5, entry_tid=15377' 2024-04-23T21:23:10.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=39, tag_tid=5, entry_tid=15527' 2024-04-23T21:23:10.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:10.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:10.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:10.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:10.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=39, tag_tid=5, entry_tid=15527' 2024-04-23T21:23:10.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:10.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:10.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:10.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:10.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:10.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:10.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:10.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=39, tag_tid=5, entry_tid=15527' 2024-04-23T21:23:10.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:10.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:10.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=15527 2024-04-23T21:23:10.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 15527 2024-04-23T21:23:10.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:10.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:10.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:10.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:10.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:10.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:10.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:10.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:10.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:10.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:10.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:10.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:10.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:10.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:10.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:10.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:10.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:10.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:10.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:10.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:10.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:10.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:10.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:10.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:10.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:10.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:10.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:10.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:10.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:10.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:10.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:10.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:10.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:10.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:10.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:10.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:10.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:10.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:10.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:10.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:10.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:10.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:10.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:10.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:10.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:10.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:10.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:10.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:10.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:10.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:10.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:10.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:10.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:10.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:10.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:10.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:10.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:10.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:10.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:10.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:10.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:10.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:10.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:10.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:10.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:10.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=39, tag_tid=5, entry_tid=15707], [object_number=38, tag_tid=5, entry_tid=15706], [object_number=37, tag_tid=5, entry_tid=15705], [object_number=36, tag_tid=5, entry_tid=15704]]], state=connected] 2024-04-23T21:23:10.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:10.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:10.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:10.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:10.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:10.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:10.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:10.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:10.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:10.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:10.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:10.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:10.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:10.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:10.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:10.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:10.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:10.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:10.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:10.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:10.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:10.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=39, tag_tid=5, entry_tid=15707], [object_number=38, tag_tid=5, entry_tid=15706], [object_number=37, tag_tid=5, entry_tid=15705], [object_number=36, tag_tid=5, entry_tid=15704]]], state=connected] 2024-04-23T21:23:10.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:10.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=39, tag_tid=5, entry_tid=15707' 2024-04-23T21:23:10.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=39, tag_tid=5, entry_tid=15707' 2024-04-23T21:23:10.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=39, tag_tid=5, entry_tid=15707' '!=' 'object_number=39, tag_tid=5, entry_tid=15527' 2024-04-23T21:23:10.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:10.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=39, tag_tid=5, entry_tid=15707' = 'object_number=39, tag_tid=5, entry_tid=15527' 2024-04-23T21:23:10.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=39, tag_tid=5, entry_tid=15707' 2024-04-23T21:23:10.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:10.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:10.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:10.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:10.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=39, tag_tid=5, entry_tid=15707' 2024-04-23T21:23:10.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:10.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:10.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:10.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:10.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:10.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:10.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=39, tag_tid=5, entry_tid=15707' 2024-04-23T21:23:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:10.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:10.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=15707 2024-04-23T21:23:10.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 15707 2024-04-23T21:23:10.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:10.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:10.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:10.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:10.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:10.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:10.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:10.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:10.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:10.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:10.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:10.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:10.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:10.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:10.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:10.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:10.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:10.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:10.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:10.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:10.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:10.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:10.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:10.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:10.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:10.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:10.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:10.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:10.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:10.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:10.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:10.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:10.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:10.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:10.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:10.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:10.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:10.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:10.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:11.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:11.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:11.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:11.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:11.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:11.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:11.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:11.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:11.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:11.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:11.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:11.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:11.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:11.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:11.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:11.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:11.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:11.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:11.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:11.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:11.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:11.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:11.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:11.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:11.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:11.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:11.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:11.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:11.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:11.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:11.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:11.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:11.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:11.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:11.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:11.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:11.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:11.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:11.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:11.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:11.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:11.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:11.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:11.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:11.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:11.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:11.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:11.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:11.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:11.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:11.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:11.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:11.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:11.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:11.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:11.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:11.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:11.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:11.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:11.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=39, tag_tid=5, entry_tid=15927], [object_number=38, tag_tid=5, entry_tid=15926], [object_number=37, tag_tid=5, entry_tid=15925], [object_number=36, tag_tid=5, entry_tid=15924]]], state=connected] 2024-04-23T21:23:11.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:11.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:11.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:11.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:11.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:11.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:11.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:11.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:11.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:11.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:11.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:11.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:11.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:11.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:11.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:11.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:11.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:11.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:11.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:11.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:11.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:11.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=39, tag_tid=5, entry_tid=15927], [object_number=38, tag_tid=5, entry_tid=15926], [object_number=37, tag_tid=5, entry_tid=15925], [object_number=36, tag_tid=5, entry_tid=15924]]], state=connected] 2024-04-23T21:23:11.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=39, tag_tid=5, entry_tid=15927' 2024-04-23T21:23:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=39, tag_tid=5, entry_tid=15927' 2024-04-23T21:23:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=39, tag_tid=5, entry_tid=15927' '!=' 'object_number=39, tag_tid=5, entry_tid=15707' 2024-04-23T21:23:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=39, tag_tid=5, entry_tid=15927' = 'object_number=39, tag_tid=5, entry_tid=15707' 2024-04-23T21:23:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=39, tag_tid=5, entry_tid=15927' 2024-04-23T21:23:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:11.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:11.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:11.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=39, tag_tid=5, entry_tid=15927' 2024-04-23T21:23:11.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:11.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:11.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:11.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:11.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:11.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:11.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:11.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=39, tag_tid=5, entry_tid=15927' 2024-04-23T21:23:11.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:11.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=15927 2024-04-23T21:23:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 15927 2024-04-23T21:23:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:11.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:11.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:11.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:11.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:11.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:11.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:11.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:11.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:11.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:11.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:11.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:11.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:11.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:11.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:11.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:11.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:11.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:11.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:11.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:11.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:11.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:11.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:11.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:11.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:11.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:11.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:11.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:11.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:11.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:11.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:11.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:11.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:11.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:11.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:11.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:11.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:11.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:11.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:11.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:11.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:11.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:11.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:11.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:11.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:11.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:11.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:11.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:11.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:11.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:11.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:11.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:11.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:11.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:11.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:11.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:11.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:11.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:11.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:11.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:11.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:11.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:11.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:11.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:11.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:11.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:11.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:11.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:11.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=38, tag_tid=5, entry_tid=16122], [object_number=37, tag_tid=5, entry_tid=16121], [object_number=36, tag_tid=5, entry_tid=16120], [object_number=39, tag_tid=5, entry_tid=16119]]], state=connected] 2024-04-23T21:23:11.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:11.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:11.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:11.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:11.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:11.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:11.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:11.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:11.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:11.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:11.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:11.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:11.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:11.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:11.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:11.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:11.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:11.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:11.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:11.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:11.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:11.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=38, tag_tid=5, entry_tid=16122], [object_number=37, tag_tid=5, entry_tid=16121], [object_number=36, tag_tid=5, entry_tid=16120], [object_number=39, tag_tid=5, entry_tid=16119]]], state=connected] 2024-04-23T21:23:11.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:11.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=38, tag_tid=5, entry_tid=16122' 2024-04-23T21:23:11.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=38, tag_tid=5, entry_tid=16122' 2024-04-23T21:23:11.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=38, tag_tid=5, entry_tid=16122' '!=' 'object_number=39, tag_tid=5, entry_tid=15927' 2024-04-23T21:23:11.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:11.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=38, tag_tid=5, entry_tid=16122' = 'object_number=39, tag_tid=5, entry_tid=15927' 2024-04-23T21:23:11.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=38, tag_tid=5, entry_tid=16122' 2024-04-23T21:23:11.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:11.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:11.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:11.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:11.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=38, tag_tid=5, entry_tid=16122' 2024-04-23T21:23:11.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:11.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:11.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:11.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:11.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:11.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:11.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:11.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=38, tag_tid=5, entry_tid=16122' 2024-04-23T21:23:11.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:11.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:11.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=16122 2024-04-23T21:23:11.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 16122 2024-04-23T21:23:11.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:11.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:11.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:11.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:11.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:11.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:11.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:11.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:11.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:11.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:11.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:11.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:11.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:11.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:12.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:12.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:12.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:12.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:12.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:12.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:12.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:12.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:12.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:12.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:12.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:12.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:12.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:12.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:12.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:12.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:12.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:12.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:12.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:12.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:12.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:12.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:12.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:12.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:12.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:12.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:12.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:12.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:12.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:12.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:12.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:12.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:12.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:12.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:12.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:12.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:12.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:12.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:12.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:12.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:12.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:12.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:12.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:12.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:12.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:12.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:12.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:12.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:12.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:12.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:12.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:12.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:12.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:12.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:12.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:12.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:12.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:12.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:12.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:12.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=38, tag_tid=5, entry_tid=16318], [object_number=37, tag_tid=5, entry_tid=16317], [object_number=36, tag_tid=5, entry_tid=16316], [object_number=39, tag_tid=5, entry_tid=16315]]], state=connected] 2024-04-23T21:23:12.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:12.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:12.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:12.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:12.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:12.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:12.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:12.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:12.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:12.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:12.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:12.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:12.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:12.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:12.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:12.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:12.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:12.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:12.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:12.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:12.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:12.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=38, tag_tid=5, entry_tid=16318], [object_number=37, tag_tid=5, entry_tid=16317], [object_number=36, tag_tid=5, entry_tid=16316], [object_number=39, tag_tid=5, entry_tid=16315]]], state=connected] 2024-04-23T21:23:12.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:12.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=38, tag_tid=5, entry_tid=16318' 2024-04-23T21:23:12.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=38, tag_tid=5, entry_tid=16318' 2024-04-23T21:23:12.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=38, tag_tid=5, entry_tid=16318' '!=' 'object_number=38, tag_tid=5, entry_tid=16122' 2024-04-23T21:23:12.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:12.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=38, tag_tid=5, entry_tid=16318' = 'object_number=38, tag_tid=5, entry_tid=16122' 2024-04-23T21:23:12.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=38, tag_tid=5, entry_tid=16318' 2024-04-23T21:23:12.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:12.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:12.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:12.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:12.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=38, tag_tid=5, entry_tid=16318' 2024-04-23T21:23:12.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:12.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:12.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:12.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:12.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:12.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:12.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:12.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=38, tag_tid=5, entry_tid=16318' 2024-04-23T21:23:12.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:12.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:12.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=16318 2024-04-23T21:23:12.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 16318 2024-04-23T21:23:12.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:12.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:12.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:12.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:12.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:12.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:12.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:12.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:12.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:12.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:12.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:12.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:12.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:12.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:12.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:12.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:12.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:12.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:12.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:12.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:12.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:12.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:12.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:12.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:12.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:12.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:12.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:12.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:12.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:12.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:12.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:12.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:12.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:12.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:12.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:12.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:12.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:12.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:12.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:12.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:12.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:12.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:12.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:12.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:12.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:12.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:12.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:12.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:12.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:12.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:12.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:12.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:12.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:12.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:12.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:12.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:12.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:12.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:12.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:12.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:12.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:12.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:12.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:12.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:12.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:12.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:12.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:12.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:12.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:12.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:12.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:12.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:12.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:12.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:12.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:12.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:12.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:12.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:12.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:12.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:12.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:12.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:12.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:12.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:12.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:12.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:12.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:12.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:12.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:12.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:12.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:12.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:12.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:12.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:12.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:12.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:12.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=37, tag_tid=5, entry_tid=16537], [object_number=36, tag_tid=5, entry_tid=16536], [object_number=39, tag_tid=5, entry_tid=16535], [object_number=38, tag_tid=5, entry_tid=16534]]], state=connected] 2024-04-23T21:23:12.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:12.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:12.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:12.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:12.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:12.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:12.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:12.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:12.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:12.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:12.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:12.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:12.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:12.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:12.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:12.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:12.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:12.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:12.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:12.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:12.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:12.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=37, tag_tid=5, entry_tid=16537], [object_number=36, tag_tid=5, entry_tid=16536], [object_number=39, tag_tid=5, entry_tid=16535], [object_number=38, tag_tid=5, entry_tid=16534]]], state=connected] 2024-04-23T21:23:12.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:12.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=37, tag_tid=5, entry_tid=16537' 2024-04-23T21:23:12.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=37, tag_tid=5, entry_tid=16537' 2024-04-23T21:23:12.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=37, tag_tid=5, entry_tid=16537' '!=' 'object_number=38, tag_tid=5, entry_tid=16318' 2024-04-23T21:23:12.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:12.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=37, tag_tid=5, entry_tid=16537' = 'object_number=38, tag_tid=5, entry_tid=16318' 2024-04-23T21:23:12.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=37, tag_tid=5, entry_tid=16537' 2024-04-23T21:23:12.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:12.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:12.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:12.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:12.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=37, tag_tid=5, entry_tid=16537' 2024-04-23T21:23:12.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:12.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:12.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:12.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:12.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:12.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:12.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:12.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=37, tag_tid=5, entry_tid=16537' 2024-04-23T21:23:12.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:12.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:12.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=16537 2024-04-23T21:23:12.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 16537 2024-04-23T21:23:12.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:12.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:12.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:13.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:13.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:13.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:13.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:13.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:13.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:13.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:13.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:13.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:13.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:13.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:13.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:13.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:13.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:13.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:13.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:13.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:13.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:13.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:13.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:13.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:13.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:13.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:13.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:13.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:13.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:13.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:13.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:13.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:13.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:13.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:13.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:13.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:13.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:13.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:13.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:13.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:13.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:13.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:13.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:13.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:13.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:13.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:13.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:13.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:13.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:13.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:13.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:13.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:13.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:13.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:13.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:13.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:13.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:13.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:13.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:13.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:13.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:13.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:13.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:13.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:13.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:13.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:13.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:13.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:13.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:13.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:13.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:13.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:13.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:13.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:13.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:13.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:13.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:13.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:13.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:13.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:13.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:13.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:13.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:13.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:13.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:13.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:13.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:13.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:13.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:13.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:13.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:13.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:13.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:13.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=38, tag_tid=5, entry_tid=16858], [object_number=37, tag_tid=5, entry_tid=16857], [object_number=36, tag_tid=5, entry_tid=16856], [object_number=39, tag_tid=5, entry_tid=16855]]], state=connected] 2024-04-23T21:23:13.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:13.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:13.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:13.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:13.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:13.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:13.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:13.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:13.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:13.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:13.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:13.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:13.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:13.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:13.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:13.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:13.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:13.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:13.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:13.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:13.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:13.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=38, tag_tid=5, entry_tid=16858], [object_number=37, tag_tid=5, entry_tid=16857], [object_number=36, tag_tid=5, entry_tid=16856], [object_number=39, tag_tid=5, entry_tid=16855]]], state=connected] 2024-04-23T21:23:13.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:13.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=38, tag_tid=5, entry_tid=16858' 2024-04-23T21:23:13.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=38, tag_tid=5, entry_tid=16858' 2024-04-23T21:23:13.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=38, tag_tid=5, entry_tid=16858' '!=' 'object_number=37, tag_tid=5, entry_tid=16537' 2024-04-23T21:23:13.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:13.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=38, tag_tid=5, entry_tid=16858' = 'object_number=37, tag_tid=5, entry_tid=16537' 2024-04-23T21:23:13.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=38, tag_tid=5, entry_tid=16858' 2024-04-23T21:23:13.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:13.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:13.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:13.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:13.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=38, tag_tid=5, entry_tid=16858' 2024-04-23T21:23:13.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:13.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:13.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:13.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:13.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:13.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:13.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:13.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=38, tag_tid=5, entry_tid=16858' 2024-04-23T21:23:13.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:13.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:13.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=16858 2024-04-23T21:23:13.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 16858 2024-04-23T21:23:13.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:13.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:13.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:13.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:13.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:13.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:13.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:13.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:13.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:13.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:13.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:13.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:13.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:13.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:13.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:13.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:13.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:13.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:13.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:13.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:13.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:13.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:13.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:13.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:13.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:13.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:13.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:13.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:13.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:13.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:13.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:13.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:13.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:13.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:13.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:13.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:13.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:13.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:13.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:13.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:13.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:13.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:13.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:13.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:13.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:13.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:13.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:13.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:13.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:13.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:13.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:13.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:13.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:13.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:13.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:13.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:13.717 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2024-04-23T21:23:13.717 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:23:13.717 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2024-04-23T21:23:13.718 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2024-04-23T21:23:13.718 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2024-04-23T21:23:13.720 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2024-04-23T21:23:13.720 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2024-04-23T21:23:13.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:13.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:13.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:13.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:13.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:13.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:13.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:13.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:13.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:13.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:13.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:13.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:13.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:13.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:13.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:13.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:13.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:13.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:13.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:13.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:13.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:13.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:13.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:13.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:13.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:13.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:13.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:13.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:13.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:13.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:13.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:13.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:13.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:13.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:13.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:13.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:13.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:13.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:13.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:13.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:13.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:13.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:13.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:13.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:13.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:13.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:13.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:13.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:13.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:13.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:13.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:13.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:13.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:13.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:13.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:13.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:13.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=39, tag_tid=5, entry_tid=17075], [object_number=38, tag_tid=5, entry_tid=17074], [object_number=37, tag_tid=5, entry_tid=17073], [object_number=36, tag_tid=5, entry_tid=17072]]], state=connected] 2024-04-23T21:23:13.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:13.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:13.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:13.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:13.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:13.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:13.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:13.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:13.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:13.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:13.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:13.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:13.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:13.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:13.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:13.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:13.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:13.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:13.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:13.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:13.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:13.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=39, tag_tid=5, entry_tid=17075], [object_number=38, tag_tid=5, entry_tid=17074], [object_number=37, tag_tid=5, entry_tid=17073], [object_number=36, tag_tid=5, entry_tid=17072]]], state=connected] 2024-04-23T21:23:13.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:13.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=39, tag_tid=5, entry_tid=17075' 2024-04-23T21:23:13.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=39, tag_tid=5, entry_tid=17075' 2024-04-23T21:23:13.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=39, tag_tid=5, entry_tid=17075' '!=' 'object_number=38, tag_tid=5, entry_tid=16858' 2024-04-23T21:23:13.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:13.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=39, tag_tid=5, entry_tid=17075' = 'object_number=38, tag_tid=5, entry_tid=16858' 2024-04-23T21:23:13.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=39, tag_tid=5, entry_tid=17075' 2024-04-23T21:23:13.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:13.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:13.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:13.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:13.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=39, tag_tid=5, entry_tid=17075' 2024-04-23T21:23:13.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:13.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:13.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:13.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:13.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:13.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:13.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:13.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=39, tag_tid=5, entry_tid=17075' 2024-04-23T21:23:13.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:13.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:13.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=17075 2024-04-23T21:23:13.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 17075 2024-04-23T21:23:13.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:13.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:13.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:14.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:14.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:14.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:14.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:14.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:14.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:14.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:14.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:14.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:14.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:14.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:14.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:14.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:14.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:14.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:14.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:14.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:14.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:14.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:14.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:14.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:14.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:14.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:14.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:14.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:14.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:14.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:14.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:14.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:14.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:14.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:14.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:14.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:14.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:14.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:14.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:14.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:14.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:14.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:14.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:14.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:14.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:14.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:14.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:14.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:14.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:14.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:14.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:14.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:14.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:14.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:14.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:14.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:14.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:14.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:14.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:14.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:14.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:14.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:14.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:14.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:14.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:14.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:14.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:14.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:14.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:14.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:14.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:14.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:14.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:14.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:14.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:14.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:14.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:14.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:14.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:14.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:14.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:14.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:14.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:14.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:14.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:14.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:14.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:14.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:14.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:14.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:14.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:14.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:14.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:14.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:14.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:14.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:14.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:14.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:14.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:14.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:14.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:14.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:14.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:14.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:14.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:14.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:14.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:14.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:14.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:14.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:14.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:14.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:14.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:14.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:14.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:14.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=36, tag_tid=5, entry_tid=17252], [object_number=39, tag_tid=5, entry_tid=17251], [object_number=38, tag_tid=5, entry_tid=17250], [object_number=37, tag_tid=5, entry_tid=17249]]], state=connected] 2024-04-23T21:23:14.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:14.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:14.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:14.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:14.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:14.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:14.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:14.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:14.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:14.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:14.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:14.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:14.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:14.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:14.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:14.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:14.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:14.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:14.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:14.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:14.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:14.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=36, tag_tid=5, entry_tid=17252], [object_number=39, tag_tid=5, entry_tid=17251], [object_number=38, tag_tid=5, entry_tid=17250], [object_number=37, tag_tid=5, entry_tid=17249]]], state=connected] 2024-04-23T21:23:14.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:14.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=36, tag_tid=5, entry_tid=17252' 2024-04-23T21:23:14.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=36, tag_tid=5, entry_tid=17252' 2024-04-23T21:23:14.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=36, tag_tid=5, entry_tid=17252' '!=' 'object_number=39, tag_tid=5, entry_tid=17075' 2024-04-23T21:23:14.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:14.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=36, tag_tid=5, entry_tid=17252' = 'object_number=39, tag_tid=5, entry_tid=17075' 2024-04-23T21:23:14.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=36, tag_tid=5, entry_tid=17252' 2024-04-23T21:23:14.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:14.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:14.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:14.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:14.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=36, tag_tid=5, entry_tid=17252' 2024-04-23T21:23:14.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:14.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:14.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:14.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:14.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:14.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:14.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:14.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=36, tag_tid=5, entry_tid=17252' 2024-04-23T21:23:14.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:14.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:14.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=17252 2024-04-23T21:23:14.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 17252 2024-04-23T21:23:14.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:14.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:14.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:14.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:14.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:14.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:14.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:14.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:14.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:14.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:14.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:14.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:14.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:14.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:14.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:14.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:14.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:14.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:14.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:14.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:14.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:14.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:14.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:14.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:14.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:14.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:14.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:14.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:14.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:14.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:14.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:14.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:14.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:14.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:14.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:14.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:14.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:14.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:14.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:14.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:14.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:14.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:14.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:14.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:14.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:14.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:14.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:14.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:14.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:14.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:14.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:14.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:14.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:14.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:14.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:14.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:14.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:14.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:14.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:14.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:14.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:14.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:14.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:14.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:14.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:14.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:14.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:14.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:14.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:14.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:14.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:14.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:14.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:14.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:14.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:14.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:14.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:14.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:14.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:14.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:14.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:14.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:14.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:14.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:14.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:14.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:14.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:14.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:14.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:14.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:14.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:14.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=39, tag_tid=5, entry_tid=17427], [object_number=38, tag_tid=5, entry_tid=17426], [object_number=37, tag_tid=5, entry_tid=17425], [object_number=36, tag_tid=5, entry_tid=17424]]], state=connected] 2024-04-23T21:23:14.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:14.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:14.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:14.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:14.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:14.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:14.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:14.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:14.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:14.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:14.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:14.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:14.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:14.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:14.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:14.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:14.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:14.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:14.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:14.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:14.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:14.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=39, tag_tid=5, entry_tid=17427], [object_number=38, tag_tid=5, entry_tid=17426], [object_number=37, tag_tid=5, entry_tid=17425], [object_number=36, tag_tid=5, entry_tid=17424]]], state=connected] 2024-04-23T21:23:14.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:14.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=39, tag_tid=5, entry_tid=17427' 2024-04-23T21:23:14.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=39, tag_tid=5, entry_tid=17427' 2024-04-23T21:23:14.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=39, tag_tid=5, entry_tid=17427' '!=' 'object_number=36, tag_tid=5, entry_tid=17252' 2024-04-23T21:23:14.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:14.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=39, tag_tid=5, entry_tid=17427' = 'object_number=36, tag_tid=5, entry_tid=17252' 2024-04-23T21:23:14.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=39, tag_tid=5, entry_tid=17427' 2024-04-23T21:23:14.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:14.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:14.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:14.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:14.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=39, tag_tid=5, entry_tid=17427' 2024-04-23T21:23:14.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:14.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:14.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:14.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:14.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:14.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:14.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:14.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=39, tag_tid=5, entry_tid=17427' 2024-04-23T21:23:14.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:14.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=17427 2024-04-23T21:23:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 17427 2024-04-23T21:23:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:14.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:15.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:15.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:15.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:15.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:15.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:15.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:15.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:15.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:15.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:15.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:15.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:15.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:15.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:15.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:15.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:15.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:15.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:15.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:15.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:15.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:15.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:15.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:15.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:15.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:15.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:15.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:15.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:15.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:15.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:15.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:15.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:15.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:15.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:15.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:15.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:15.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:15.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:15.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:15.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:15.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:15.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:15.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:15.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:15.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:15.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:15.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:15.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:15.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:15.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:15.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:15.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:15.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:15.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:15.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:15.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:15.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:15.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:15.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:15.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:15.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:15.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:15.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:15.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:15.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:15.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:15.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:15.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:15.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:15.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:15.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:15.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:15.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:15.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:15.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:15.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:15.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=39, tag_tid=5, entry_tid=17631], [object_number=38, tag_tid=5, entry_tid=17630], [object_number=37, tag_tid=5, entry_tid=17629], [object_number=36, tag_tid=5, entry_tid=17628]]], state=connected] 2024-04-23T21:23:15.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:15.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:15.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:15.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:15.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:15.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:15.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:15.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:15.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:15.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:15.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:15.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:15.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:15.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:15.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:15.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:15.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:15.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:15.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:15.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:15.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:15.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=39, tag_tid=5, entry_tid=17631], [object_number=38, tag_tid=5, entry_tid=17630], [object_number=37, tag_tid=5, entry_tid=17629], [object_number=36, tag_tid=5, entry_tid=17628]]], state=connected] 2024-04-23T21:23:15.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:15.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=39, tag_tid=5, entry_tid=17631' 2024-04-23T21:23:15.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=39, tag_tid=5, entry_tid=17631' 2024-04-23T21:23:15.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=39, tag_tid=5, entry_tid=17631' '!=' 'object_number=39, tag_tid=5, entry_tid=17427' 2024-04-23T21:23:15.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:15.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=39, tag_tid=5, entry_tid=17631' = 'object_number=39, tag_tid=5, entry_tid=17427' 2024-04-23T21:23:15.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=39, tag_tid=5, entry_tid=17631' 2024-04-23T21:23:15.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:15.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:15.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:15.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:15.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=39, tag_tid=5, entry_tid=17631' 2024-04-23T21:23:15.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:15.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:15.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:15.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:15.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:15.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:15.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:15.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=39, tag_tid=5, entry_tid=17631' 2024-04-23T21:23:15.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:15.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:15.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=17631 2024-04-23T21:23:15.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 17631 2024-04-23T21:23:15.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:15.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:15.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:15.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:15.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:15.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:15.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:15.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:15.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:15.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:15.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:15.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:15.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:15.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:15.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:15.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:15.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:15.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:15.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:15.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:15.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:15.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:15.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:15.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:15.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:15.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:15.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:15.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:15.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:15.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:15.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:15.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:15.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:15.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:15.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:15.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:15.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:15.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:15.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:15.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:15.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:15.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:15.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:15.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:15.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:15.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:15.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:15.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:15.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:15.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:15.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:15.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:15.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:15.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:15.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:15.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:15.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:15.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:15.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:15.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:15.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:15.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:15.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:15.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:15.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:15.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:15.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=39, tag_tid=5, entry_tid=17831], [object_number=38, tag_tid=5, entry_tid=17830], [object_number=37, tag_tid=5, entry_tid=17829], [object_number=36, tag_tid=5, entry_tid=17828]]], state=connected] 2024-04-23T21:23:15.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:15.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:15.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:15.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:15.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:15.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:15.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:15.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:15.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:15.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:15.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:15.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:15.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:15.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:15.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:15.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:15.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:16.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:16.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:16.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:16.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:16.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=39, tag_tid=5, entry_tid=17831], [object_number=38, tag_tid=5, entry_tid=17830], [object_number=37, tag_tid=5, entry_tid=17829], [object_number=36, tag_tid=5, entry_tid=17828]]], state=connected] 2024-04-23T21:23:16.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:16.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=39, tag_tid=5, entry_tid=17831' 2024-04-23T21:23:16.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=39, tag_tid=5, entry_tid=17831' 2024-04-23T21:23:16.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=39, tag_tid=5, entry_tid=17831' '!=' 'object_number=39, tag_tid=5, entry_tid=17631' 2024-04-23T21:23:16.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:16.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=39, tag_tid=5, entry_tid=17831' = 'object_number=39, tag_tid=5, entry_tid=17631' 2024-04-23T21:23:16.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=39, tag_tid=5, entry_tid=17831' 2024-04-23T21:23:16.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:16.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:16.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:16.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:16.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=39, tag_tid=5, entry_tid=17831' 2024-04-23T21:23:16.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:16.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:16.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:16.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:16.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:16.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:16.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:16.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=39, tag_tid=5, entry_tid=17831' 2024-04-23T21:23:16.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:16.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:16.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=17831 2024-04-23T21:23:16.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 17831 2024-04-23T21:23:16.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:16.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:16.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:16.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:16.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:16.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:16.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:16.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:16.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:16.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:16.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:16.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:16.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:16.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:16.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:16.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:16.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:16.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:16.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:16.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:16.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:16.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:16.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:16.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:16.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:16.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:16.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:16.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:16.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:16.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:16.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:16.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:16.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:16.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:16.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:16.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:16.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:16.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:16.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:16.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:16.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:16.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:16.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:16.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:16.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:16.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:16.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:16.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:16.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:16.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:16.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:16.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:16.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:16.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:16.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:16.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:16.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:16.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:16.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:16.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:16.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:16.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:16.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:16.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:16.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:16.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:16.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:16.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:16.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:16.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:16.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:16.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:16.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:16.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:16.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:16.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:16.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:16.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:16.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:16.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:16.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:16.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:16.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=38, tag_tid=5, entry_tid=18038], [object_number=37, tag_tid=5, entry_tid=18037], [object_number=36, tag_tid=5, entry_tid=18036], [object_number=39, tag_tid=5, entry_tid=18035]]], state=connected] 2024-04-23T21:23:16.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:16.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:16.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:16.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:16.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:16.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:16.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:16.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:16.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:16.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:16.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:16.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:16.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:16.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:16.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:16.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:16.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:16.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=38, tag_tid=5, entry_tid=18038], [object_number=37, tag_tid=5, entry_tid=18037], [object_number=36, tag_tid=5, entry_tid=18036], [object_number=39, tag_tid=5, entry_tid=18035]]], state=connected] 2024-04-23T21:23:16.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:16.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=38, tag_tid=5, entry_tid=18038' 2024-04-23T21:23:16.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=38, tag_tid=5, entry_tid=18038' 2024-04-23T21:23:16.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=38, tag_tid=5, entry_tid=18038' '!=' 'object_number=39, tag_tid=5, entry_tid=17831' 2024-04-23T21:23:16.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:16.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=38, tag_tid=5, entry_tid=18038' = 'object_number=39, tag_tid=5, entry_tid=17831' 2024-04-23T21:23:16.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=38, tag_tid=5, entry_tid=18038' 2024-04-23T21:23:16.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:16.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:16.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:16.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:16.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=38, tag_tid=5, entry_tid=18038' 2024-04-23T21:23:16.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:16.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:16.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:16.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:16.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:16.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:16.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:16.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=38, tag_tid=5, entry_tid=18038' 2024-04-23T21:23:16.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:16.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:16.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=18038 2024-04-23T21:23:16.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 18038 2024-04-23T21:23:16.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:16.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:16.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:16.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:16.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:16.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:16.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:16.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:16.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:16.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:16.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:16.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:16.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:16.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:16.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:16.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:16.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:16.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:16.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:16.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:16.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:16.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:16.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:16.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:16.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:16.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:16.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:16.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:16.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:16.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:16.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:16.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:16.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:16.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:16.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:16.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:16.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:16.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:16.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:16.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:16.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:16.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:16.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:16.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:16.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:16.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:16.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:16.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:16.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:16.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:16.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:16.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:16.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:16.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:16.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:16.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:16.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:16.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:16.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:16.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:16.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:16.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:16.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:16.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:16.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:16.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:16.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:16.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:16.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:16.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:16.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:16.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:16.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:16.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:16.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:16.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:16.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:16.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:16.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:16.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:16.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:16.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:16.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:16.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:16.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:16.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:16.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:16.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=39, tag_tid=5, entry_tid=18247], [object_number=38, tag_tid=5, entry_tid=18246], [object_number=37, tag_tid=5, entry_tid=18245], [object_number=36, tag_tid=5, entry_tid=18244]]], state=connected] 2024-04-23T21:23:16.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:16.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:16.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:16.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:16.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:16.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:16.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:16.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:16.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:16.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:16.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:16.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:16.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:16.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:16.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:16.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:17.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:17.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:17.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:17.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:17.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=39, tag_tid=5, entry_tid=18247], [object_number=38, tag_tid=5, entry_tid=18246], [object_number=37, tag_tid=5, entry_tid=18245], [object_number=36, tag_tid=5, entry_tid=18244]]], state=connected] 2024-04-23T21:23:17.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:17.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=39, tag_tid=5, entry_tid=18247' 2024-04-23T21:23:17.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=39, tag_tid=5, entry_tid=18247' 2024-04-23T21:23:17.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=39, tag_tid=5, entry_tid=18247' '!=' 'object_number=38, tag_tid=5, entry_tid=18038' 2024-04-23T21:23:17.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:17.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=39, tag_tid=5, entry_tid=18247' = 'object_number=38, tag_tid=5, entry_tid=18038' 2024-04-23T21:23:17.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=39, tag_tid=5, entry_tid=18247' 2024-04-23T21:23:17.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:17.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:17.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:17.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:17.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=39, tag_tid=5, entry_tid=18247' 2024-04-23T21:23:17.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:17.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:17.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:17.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:17.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:17.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:17.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:17.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=39, tag_tid=5, entry_tid=18247' 2024-04-23T21:23:17.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:17.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:17.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=18247 2024-04-23T21:23:17.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 18247 2024-04-23T21:23:17.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:17.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:17.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:17.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:17.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:17.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:17.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:17.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:17.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:17.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:17.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:17.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:17.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:17.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:17.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:17.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:17.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:17.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:17.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:17.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:17.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:17.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:17.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:17.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:17.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:17.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:17.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:17.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:17.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:17.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:17.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:17.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:17.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:17.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:17.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:17.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:17.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:17.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:17.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:17.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:17.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:17.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:17.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:17.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:17.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:17.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:17.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:17.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:17.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:17.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:17.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:17.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:17.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:17.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:17.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:17.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:17.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:17.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:17.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:17.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:17.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:17.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:17.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:17.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:17.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:17.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:17.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:17.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:17.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:17.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:17.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:17.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:17.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:17.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:17.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:17.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:17.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:17.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:17.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:17.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:17.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:17.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:17.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:17.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:17.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:17.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:17.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:17.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:17.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:17.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:17.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:17.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:17.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:17.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:17.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:17.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:17.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:17.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:17.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:17.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:17.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:17.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:17.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=37, tag_tid=5, entry_tid=18461], [object_number=36, tag_tid=5, entry_tid=18460], [object_number=39, tag_tid=5, entry_tid=18459], [object_number=38, tag_tid=5, entry_tid=18458]]], state=connected] 2024-04-23T21:23:17.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:17.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:17.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:17.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:17.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:17.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:17.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:17.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:17.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:17.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:17.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:17.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:17.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:17.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:17.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:17.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:17.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:17.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:17.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:17.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:17.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:17.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=37, tag_tid=5, entry_tid=18461], [object_number=36, tag_tid=5, entry_tid=18460], [object_number=39, tag_tid=5, entry_tid=18459], [object_number=38, tag_tid=5, entry_tid=18458]]], state=connected] 2024-04-23T21:23:17.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:17.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=37, tag_tid=5, entry_tid=18461' 2024-04-23T21:23:17.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=37, tag_tid=5, entry_tid=18461' 2024-04-23T21:23:17.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=37, tag_tid=5, entry_tid=18461' '!=' 'object_number=39, tag_tid=5, entry_tid=18247' 2024-04-23T21:23:17.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:17.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=37, tag_tid=5, entry_tid=18461' = 'object_number=39, tag_tid=5, entry_tid=18247' 2024-04-23T21:23:17.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=37, tag_tid=5, entry_tid=18461' 2024-04-23T21:23:17.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:17.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:17.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:17.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:17.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=37, tag_tid=5, entry_tid=18461' 2024-04-23T21:23:17.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:17.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:17.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:17.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:17.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:17.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:17.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:17.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=37, tag_tid=5, entry_tid=18461' 2024-04-23T21:23:17.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:17.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:17.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=18461 2024-04-23T21:23:17.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 18461 2024-04-23T21:23:17.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:17.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:17.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:17.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:17.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:17.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:17.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:17.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:17.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:17.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:17.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:17.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:17.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:17.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:17.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:17.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:17.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:17.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:17.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:17.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:17.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:17.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:17.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:17.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:17.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:17.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:17.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:17.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:17.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:17.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:17.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:17.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:17.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:17.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:17.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:17.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:17.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:17.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:17.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:17.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:17.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:17.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:17.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:17.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:17.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:17.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:17.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:17.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:17.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:17.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:17.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:17.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:17.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:17.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:17.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:17.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:17.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:17.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:17.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:17.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:17.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:17.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:17.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:17.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:17.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:17.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:17.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:17.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:17.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:17.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:17.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:17.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:17.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:17.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:17.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:17.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:17.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:17.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:17.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:17.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:17.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:17.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:17.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:17.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:17.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:17.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:17.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:17.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:17.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:17.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:17.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:17.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:17.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:17.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:17.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:17.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:17.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:17.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:17.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:17.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:17.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:17.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=36, tag_tid=5, entry_tid=18668], [object_number=39, tag_tid=5, entry_tid=18667], [object_number=38, tag_tid=5, entry_tid=18666], [object_number=37, tag_tid=5, entry_tid=18665]]], state=connected] 2024-04-23T21:23:17.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:17.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:17.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:17.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:17.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:17.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:17.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:17.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:17.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:17.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:17.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:17.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:17.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:17.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:18.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:18.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:18.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:18.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:18.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:18.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:18.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:18.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=36, tag_tid=5, entry_tid=18668], [object_number=39, tag_tid=5, entry_tid=18667], [object_number=38, tag_tid=5, entry_tid=18666], [object_number=37, tag_tid=5, entry_tid=18665]]], state=connected] 2024-04-23T21:23:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:18.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=36, tag_tid=5, entry_tid=18668' 2024-04-23T21:23:18.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=36, tag_tid=5, entry_tid=18668' 2024-04-23T21:23:18.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=36, tag_tid=5, entry_tid=18668' '!=' 'object_number=37, tag_tid=5, entry_tid=18461' 2024-04-23T21:23:18.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:18.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=36, tag_tid=5, entry_tid=18668' = 'object_number=37, tag_tid=5, entry_tid=18461' 2024-04-23T21:23:18.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=36, tag_tid=5, entry_tid=18668' 2024-04-23T21:23:18.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:18.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:18.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:18.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:18.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=36, tag_tid=5, entry_tid=18668' 2024-04-23T21:23:18.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:18.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:18.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:18.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:18.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:18.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:18.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:18.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=36, tag_tid=5, entry_tid=18668' 2024-04-23T21:23:18.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:18.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:18.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=18668 2024-04-23T21:23:18.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 18668 2024-04-23T21:23:18.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:18.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:18.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:18.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:18.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:18.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:18.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:18.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:18.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:18.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:18.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:18.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:18.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:18.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:18.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:18.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:18.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:18.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:18.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:18.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:18.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:18.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:18.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:18.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:18.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:18.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:18.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:18.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:18.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:18.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:18.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:18.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:18.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:18.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:18.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:18.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:18.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:18.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:18.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:18.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:18.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:18.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:18.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:18.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:18.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:18.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:18.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:18.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:18.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:18.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:18.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:18.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:18.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:18.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:18.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:18.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:18.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:18.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:18.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:18.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:18.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:18.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:18.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:18.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:18.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:18.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:18.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:18.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:18.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:18.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:18.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:18.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:18.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:18.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:18.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:18.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:18.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:18.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:18.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:18.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:18.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:18.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:18.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:18.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:18.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:18.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:18.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:18.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=37, tag_tid=5, entry_tid=18905], [object_number=36, tag_tid=5, entry_tid=18904], [object_number=39, tag_tid=5, entry_tid=18903], [object_number=38, tag_tid=5, entry_tid=18902]]], state=connected] 2024-04-23T21:23:18.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:18.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:18.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:18.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:18.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:18.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:18.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:18.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:18.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:18.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:18.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:18.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:18.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:18.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:18.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:18.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:18.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:18.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:18.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:18.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:18.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:18.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=37, tag_tid=5, entry_tid=18905], [object_number=36, tag_tid=5, entry_tid=18904], [object_number=39, tag_tid=5, entry_tid=18903], [object_number=38, tag_tid=5, entry_tid=18902]]], state=connected] 2024-04-23T21:23:18.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:18.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=37, tag_tid=5, entry_tid=18905' 2024-04-23T21:23:18.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=37, tag_tid=5, entry_tid=18905' 2024-04-23T21:23:18.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=37, tag_tid=5, entry_tid=18905' '!=' 'object_number=36, tag_tid=5, entry_tid=18668' 2024-04-23T21:23:18.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:18.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=37, tag_tid=5, entry_tid=18905' = 'object_number=36, tag_tid=5, entry_tid=18668' 2024-04-23T21:23:18.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=37, tag_tid=5, entry_tid=18905' 2024-04-23T21:23:18.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:18.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:18.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:18.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=37, tag_tid=5, entry_tid=18905' 2024-04-23T21:23:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:18.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:18.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:18.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:18.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:18.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:18.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=37, tag_tid=5, entry_tid=18905' 2024-04-23T21:23:18.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:18.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:18.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=18905 2024-04-23T21:23:18.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 18905 2024-04-23T21:23:18.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:18.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:18.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:18.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:18.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:18.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:18.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:18.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:18.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:18.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:18.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:18.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:18.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:18.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:18.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:18.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:18.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:18.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:18.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:18.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:18.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:18.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:18.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:18.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:18.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:18.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:18.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:18.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:18.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:18.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:18.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:18.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:18.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:18.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:18.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:18.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:18.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:18.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:18.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:18.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:18.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:18.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:18.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:18.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:18.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:18.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:18.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:18.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:18.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:18.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:18.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:18.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:18.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:18.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:18.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:18.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:18.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:18.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:18.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:18.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:18.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:18.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:18.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:18.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:18.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:18.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:18.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:18.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:18.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:18.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:18.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:18.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:18.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:18.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:18.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:18.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:18.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:18.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:18.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:18.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:18.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:18.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:18.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:18.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:18.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:18.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:18.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:18.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:18.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:18.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:18.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:18.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:19.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:19.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:19.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:19.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:19.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=38, tag_tid=5, entry_tid=19142], [object_number=37, tag_tid=5, entry_tid=19141], [object_number=36, tag_tid=5, entry_tid=19140], [object_number=39, tag_tid=5, entry_tid=19139]]], state=connected] 2024-04-23T21:23:19.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:19.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:19.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:19.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:19.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:19.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:19.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:19.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:19.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:19.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:19.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:19.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:19.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:19.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=38, tag_tid=5, entry_tid=19142], [object_number=37, tag_tid=5, entry_tid=19141], [object_number=36, tag_tid=5, entry_tid=19140], [object_number=39, tag_tid=5, entry_tid=19139]]], state=connected] 2024-04-23T21:23:19.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:19.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=38, tag_tid=5, entry_tid=19142' 2024-04-23T21:23:19.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=38, tag_tid=5, entry_tid=19142' 2024-04-23T21:23:19.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=38, tag_tid=5, entry_tid=19142' '!=' 'object_number=37, tag_tid=5, entry_tid=18905' 2024-04-23T21:23:19.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:19.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=38, tag_tid=5, entry_tid=19142' = 'object_number=37, tag_tid=5, entry_tid=18905' 2024-04-23T21:23:19.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=38, tag_tid=5, entry_tid=19142' 2024-04-23T21:23:19.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:19.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:19.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:19.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:19.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=38, tag_tid=5, entry_tid=19142' 2024-04-23T21:23:19.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:19.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:19.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:19.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:19.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:19.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:19.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:19.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=38, tag_tid=5, entry_tid=19142' 2024-04-23T21:23:19.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:19.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:19.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=19142 2024-04-23T21:23:19.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 19142 2024-04-23T21:23:19.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:19.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:19.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:19.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:19.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:19.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:19.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:19.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:19.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:19.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:19.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:19.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:19.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:19.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:19.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:19.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:19.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:19.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:19.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:19.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:19.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:19.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:19.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:19.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:19.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:19.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:19.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:19.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:19.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:19.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:19.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:19.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:19.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:19.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:19.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:19.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:19.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:19.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:19.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:19.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:19.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:19.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:19.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:19.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:19.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:19.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:19.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:19.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:19.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:19.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:19.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:19.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:19.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:19.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:19.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:19.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:19.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:19.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:19.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:19.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:19.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:19.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:19.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:19.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:19.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:19.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:19.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:19.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:19.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:19.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:19.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:19.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:19.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:19.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:19.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:19.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:19.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:19.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:19.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:19.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:19.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:19.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:19.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:19.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:19.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:19.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:19.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:19.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:19.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:19.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:19.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:19.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:19.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:19.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:19.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:19.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:19.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:19.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:19.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:19.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:19.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:19.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:19.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:19.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:19.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:19.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:19.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:19.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:19.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:19.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:19.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:19.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:19.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:19.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:19.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:19.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=37, tag_tid=5, entry_tid=19369], [object_number=36, tag_tid=5, entry_tid=19368], [object_number=39, tag_tid=5, entry_tid=19367], [object_number=38, tag_tid=5, entry_tid=19366]]], state=connected] 2024-04-23T21:23:19.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:19.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:19.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:19.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:19.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:19.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:19.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:19.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:19.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=37, tag_tid=5, entry_tid=19369], [object_number=36, tag_tid=5, entry_tid=19368], [object_number=39, tag_tid=5, entry_tid=19367], [object_number=38, tag_tid=5, entry_tid=19366]]], state=connected] 2024-04-23T21:23:19.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:19.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=37, tag_tid=5, entry_tid=19369' 2024-04-23T21:23:19.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=37, tag_tid=5, entry_tid=19369' 2024-04-23T21:23:19.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=37, tag_tid=5, entry_tid=19369' '!=' 'object_number=38, tag_tid=5, entry_tid=19142' 2024-04-23T21:23:19.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:19.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=37, tag_tid=5, entry_tid=19369' = 'object_number=38, tag_tid=5, entry_tid=19142' 2024-04-23T21:23:19.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=37, tag_tid=5, entry_tid=19369' 2024-04-23T21:23:19.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:19.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:19.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:19.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:19.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=37, tag_tid=5, entry_tid=19369' 2024-04-23T21:23:19.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:19.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:19.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:19.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:19.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:19.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:19.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:19.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=37, tag_tid=5, entry_tid=19369' 2024-04-23T21:23:19.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:19.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:19.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=19369 2024-04-23T21:23:19.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 19369 2024-04-23T21:23:19.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:19.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:19.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:19.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:19.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:19.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:19.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:19.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:19.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:19.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:19.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:19.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:19.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:19.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:19.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:19.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:19.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:19.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:19.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:19.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:19.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:19.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:19.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:19.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:19.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:19.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:19.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:19.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:19.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:19.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:19.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:19.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:19.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:19.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:19.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:19.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:19.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:19.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:19.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:19.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:19.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:19.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:19.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:19.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:19.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:19.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:19.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:19.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:19.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:19.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:19.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:19.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:19.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:19.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:19.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:19.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:19.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:19.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:19.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:19.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:19.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:19.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:19.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:19.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:19.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:19.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:19.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:19.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:19.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:19.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:19.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:19.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:19.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:19.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:19.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:19.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:19.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:19.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:19.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:19.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:19.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:19.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:19.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:19.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:19.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:19.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:19.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:19.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:19.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:19.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:19.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:19.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:19.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:19.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:19.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:19.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:19.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:20.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:20.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:20.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:20.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:20.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=36, tag_tid=5, entry_tid=19604], [object_number=39, tag_tid=5, entry_tid=19603], [object_number=38, tag_tid=5, entry_tid=19602], [object_number=37, tag_tid=5, entry_tid=19601]]], state=connected] 2024-04-23T21:23:20.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:20.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:20.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:20.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:20.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:20.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:20.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:20.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:20.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:20.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:20.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:20.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:20.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:20.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:20.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:20.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:20.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 9 2024-04-23T21:23:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=36, tag_tid=5, entry_tid=19604], [object_number=39, tag_tid=5, entry_tid=19603], [object_number=38, tag_tid=5, entry_tid=19602], [object_number=37, tag_tid=5, entry_tid=19601]]], state=connected] 2024-04-23T21:23:20.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=36, tag_tid=5, entry_tid=19604' 2024-04-23T21:23:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=36, tag_tid=5, entry_tid=19604' 2024-04-23T21:23:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=36, tag_tid=5, entry_tid=19604' '!=' 'object_number=37, tag_tid=5, entry_tid=19369' 2024-04-23T21:23:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=36, tag_tid=5, entry_tid=19604' = 'object_number=37, tag_tid=5, entry_tid=19369' 2024-04-23T21:23:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=36, tag_tid=5, entry_tid=19604' 2024-04-23T21:23:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:20.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:20.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:20.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=36, tag_tid=5, entry_tid=19604' 2024-04-23T21:23:20.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:20.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:20.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:20.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:20.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:20.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:20.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:20.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=36, tag_tid=5, entry_tid=19604' 2024-04-23T21:23:20.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:20.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:20.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=19604 2024-04-23T21:23:20.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 19604 2024-04-23T21:23:20.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:20.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:20.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:20.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:20.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:20.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:20.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:20.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:20.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:20.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:20.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:20.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:20.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:20.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:20.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:20.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:20.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:20.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:20.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:20.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:20.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:20.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:20.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:20.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:20.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:20.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:20.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:20.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:20.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:20.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:20.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:20.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:20.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:20.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:20.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:20.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:20.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:20.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:20.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:20.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:20.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:20.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:20.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:20.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:20.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:20.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:20.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:20.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:20.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:20.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:20.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:20.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:20.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:20.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:20.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:20.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:20.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:20.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:20.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:20.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:20.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:20.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:20.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:20.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:20.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:20.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:20.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:20.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:20.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:20.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:20.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:20.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 10 2024-04-23T21:23:20.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:20.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:20.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:20.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=40, tag_tid=5, entry_tid=19848], [object_number=43, tag_tid=5, entry_tid=19847], [object_number=42, tag_tid=5, entry_tid=19846], [object_number=41, tag_tid=5, entry_tid=19845]]], state=connected] 2024-04-23T21:23:20.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:20.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:20.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:20.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:20.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:20.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:20.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:20.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:20.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:20.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:20.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:20.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:20.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:20.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:20.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:20.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:20.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:20.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 10 2024-04-23T21:23:20.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:20.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:20.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:20.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=40, tag_tid=5, entry_tid=19848], [object_number=43, tag_tid=5, entry_tid=19847], [object_number=42, tag_tid=5, entry_tid=19846], [object_number=41, tag_tid=5, entry_tid=19845]]], state=connected] 2024-04-23T21:23:20.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:20.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=40, tag_tid=5, entry_tid=19848' 2024-04-23T21:23:20.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=40, tag_tid=5, entry_tid=19848' 2024-04-23T21:23:20.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=40, tag_tid=5, entry_tid=19848' '!=' 'object_number=36, tag_tid=5, entry_tid=19604' 2024-04-23T21:23:20.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:20.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=40, tag_tid=5, entry_tid=19848' = 'object_number=36, tag_tid=5, entry_tid=19604' 2024-04-23T21:23:20.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=40, tag_tid=5, entry_tid=19848' 2024-04-23T21:23:20.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:20.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:20.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:20.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:20.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=40, tag_tid=5, entry_tid=19848' 2024-04-23T21:23:20.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:20.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:20.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:20.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:20.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:20.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:20.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:20.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=40, tag_tid=5, entry_tid=19848' 2024-04-23T21:23:20.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:20.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:20.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=19848 2024-04-23T21:23:20.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 19848 2024-04-23T21:23:20.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:20.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:20.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:20.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:20.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:20.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:20.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:20.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:20.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:20.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:20.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:20.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:20.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:20.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:20.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:20.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:20.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:20.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:20.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:20.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:20.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:20.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:20.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:20.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:20.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:20.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:20.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:20.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:20.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:20.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:20.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:20.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:20.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:20.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:20.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:20.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:20.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:20.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:20.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:20.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:20.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:20.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:20.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:20.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:20.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:21.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:21.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 10 2024-04-23T21:23:21.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:21.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:21.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:21.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=41, tag_tid=5, entry_tid=19969], [object_number=40, tag_tid=5, entry_tid=19968], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:21.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:21.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:21.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:21.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:21.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:21.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:21.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:21.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:21.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 10 2024-04-23T21:23:21.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:21.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:21.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:21.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=41, tag_tid=5, entry_tid=19969], [object_number=40, tag_tid=5, entry_tid=19968], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:21.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:21.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:21.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:21.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=41, tag_tid=5, entry_tid=19969' '!=' 'object_number=40, tag_tid=5, entry_tid=19848' 2024-04-23T21:23:21.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:23:21.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=41, tag_tid=5, entry_tid=19969' = 'object_number=40, tag_tid=5, entry_tid=19848' 2024-04-23T21:23:21.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:21.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:21.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:21.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:21.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=6 2024-04-23T21:23:21.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:21.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:23:21.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:21.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=5 2024-04-23T21:23:21.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:21.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:21.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:21.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:23:21.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:21.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:23:21.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:23:21.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=19969 2024-04-23T21:23:21.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 6 = 5 -a 1 -le 19969 2024-04-23T21:23:21.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:23:21.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:21.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:23:21.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:21.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:21.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:21.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:21.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:21.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:21.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:21.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:21.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:21.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:21.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:21.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:21.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:21.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:21.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:21.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:21.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:21.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:21.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:21.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:21.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:21.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:21.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:21.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:21.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:21.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:21.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:21.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:21.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:21.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:21.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:21.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:21.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:21.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:21.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:21.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:21.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:21.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:21.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:21.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:21.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:21.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:21.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:21.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:21.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:21.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:21.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:21.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:21.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:21.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:21.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:21.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:21.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:21.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:21.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 10 2024-04-23T21:23:21.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:21.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:21.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:21.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=41, tag_tid=5, entry_tid=19969], [object_number=40, tag_tid=5, entry_tid=19968], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:21.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:21.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:21.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:21.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:21.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:21.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:21.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:21.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:21.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:21.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:21.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:21.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:21.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:21.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:21.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:21.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:21.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:21.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 10 2024-04-23T21:23:21.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:21.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:21.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:21.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=41, tag_tid=5, entry_tid=19969], [object_number=40, tag_tid=5, entry_tid=19968], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:21.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:21.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:21.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:21.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=41, tag_tid=5, entry_tid=19969' '!=' 'object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:21.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:21.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:23:22.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:22.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:22.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:22.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:22.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:22.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:22.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:22.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:22.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:22.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:22.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:22.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:22.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:22.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:22.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:22.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:22.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:22.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:22.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:22.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:22.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:22.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:22.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:22.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:22.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:22.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:22.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:22.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:22.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:22.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:22.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:22.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:22.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:22.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:22.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:22.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:22.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:22.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:22.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:22.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:22.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:22.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:22.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:22.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:22.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:22.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:22.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:22.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:22.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:22.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:22.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:22.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:22.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:22.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:22.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:22.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:22.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:22.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:22.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:22.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:22.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:22.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:22.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:22.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:22.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:22.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:22.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:22.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:22.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:22.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:22.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:22.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:22.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:22.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:22.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:22.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:22.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:22.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:22.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:22.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:22.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:22.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:22.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:22.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:22.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:22.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:22.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 10 2024-04-23T21:23:22.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:22.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:22.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:22.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=41, tag_tid=5, entry_tid=19969], [object_number=40, tag_tid=5, entry_tid=19968], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:22.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:22.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:22.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:22.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:22.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:22.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:22.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:22.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:22.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 10 2024-04-23T21:23:22.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:22.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:22.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:22.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=41, tag_tid=5, entry_tid=19969], [object_number=40, tag_tid=5, entry_tid=19968], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:22.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:22.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:22.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:22.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=41, tag_tid=5, entry_tid=19969' '!=' 'object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:22.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:22.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:23:23.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:23.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:23.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:23.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:23.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:23.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:23.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:23.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:23.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:23.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:23.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:23.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:23.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:23.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:23.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:23.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:23.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:23.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:23.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:23.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:23.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:23.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:23.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:23.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:23.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:23.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:23.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:23.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:23.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:23.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:23.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:23.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:23.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:23.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:23.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:23.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:23.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:23.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:23.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:23.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:23.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:23.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:23.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:23.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:23.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:23.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:23.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:23.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:23.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:23.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:23.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:23.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:23.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:23.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:23.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:23.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:23.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:23.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:23.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:23.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:23.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:23.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:23.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:23.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:23.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:23.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:23.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:23.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:23.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:23.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 10 2024-04-23T21:23:23.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:23.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:23.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:23.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=41, tag_tid=5, entry_tid=19969], [object_number=40, tag_tid=5, entry_tid=19968], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:23.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:23.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:23.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:23.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:23.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:23.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:23.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:23.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:23.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:23.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:23.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:23.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:23.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:23.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:23.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:23.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:23.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:23.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 10 2024-04-23T21:23:23.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:23.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:23.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:23.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=41, tag_tid=5, entry_tid=19969], [object_number=40, tag_tid=5, entry_tid=19968], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:23.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:23.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:23.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:23.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=41, tag_tid=5, entry_tid=19969' '!=' 'object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:23.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:23.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:23:25.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:25.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:25.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:25.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:25.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:25.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:25.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:25.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:25.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:25.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:25.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:25.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:25.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:25.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:25.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:25.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:25.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:25.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:25.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:25.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:25.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:25.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:25.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:25.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:25.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:25.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:25.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:25.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:25.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:25.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:25.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:25.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:25.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:25.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:25.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:25.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:25.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:25.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:25.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:25.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:25.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:25.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:25.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:25.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:25.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:25.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:25.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:25.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:25.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:25.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:25.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:25.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:25.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:25.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:25.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:25.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:25.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:25.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:25.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 10 2024-04-23T21:23:25.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:25.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:25.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:25.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=41, tag_tid=5, entry_tid=19969], [object_number=40, tag_tid=5, entry_tid=19968], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:25.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:25.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:25.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:25.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:25.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:25.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:25.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:25.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:25.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:25.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:25.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:25.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:25.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:25.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:25.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:25.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:25.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:25.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 10 2024-04-23T21:23:25.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:25.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:25.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:25.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=41, tag_tid=5, entry_tid=19969], [object_number=40, tag_tid=5, entry_tid=19968], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:25.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:25.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:25.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:25.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=41, tag_tid=5, entry_tid=19969' '!=' 'object_number=41, tag_tid=5, entry_tid=19969' 2024-04-23T21:23:25.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:23:25.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:23:27.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:23:27.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:27.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:27.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:23:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:23:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:23:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:23:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:23:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:23:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:23:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:23:27.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:23:27.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:23:27.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:27.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:23:27.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:23:27.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:27.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:27.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:23:27.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:23:27.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:27.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:27.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:27.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:23:27.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:27.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:27.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:27.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:23:27.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:27.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:23:27.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:23:27.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:27.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:27.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:23:27.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:23:27.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:27.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:27.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:27.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:23:27.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:23:27.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:27.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:27.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:23:27.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:23:27.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:23:27.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:27.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:27.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:23:27.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:23:27.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:23:27.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:23:27.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:23:27.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:23:27.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:23:27.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:23:27.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:23:27.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:23:27.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:23:27.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:23:27.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:23:27.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:23:27.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:23:27.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:23:27.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:27.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:23:27.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:27.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:23:27.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:27.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:23:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:23:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:23:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:23:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:23:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:23:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:23:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:23:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:23:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:23:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:23:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:23:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:23:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:23:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:23:27.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:23:27.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:27.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:27.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:23:27.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:23:27.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:27.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:23:27.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:27.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:27.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:27.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:23:27.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:27.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:27.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:27.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:23:27.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:27.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:27.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:27.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:27.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:27.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:27.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 10 2024-04-23T21:23:27.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:27.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:27.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:27.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:27.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:27.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:27.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:23:27.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:27.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:27.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:27.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:23:27.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:23:27.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:23:27.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:23:27.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:23:27.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:23:27.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:23:27.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:23:27.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:27.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:23:27.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:27.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 10 2024-04-23T21:23:27.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 10 2024-04-23T21:23:27.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:23:27.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:27.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=41, tag_tid=6, entry_tid=1], [object_number=40, tag_tid=6, entry_tid=0], [object_number=43, tag_tid=5, entry_tid=19967], [object_number=42, tag_tid=5, entry_tid=19966]]], state=connected] 2024-04-23T21:23:27.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:23:27.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:27.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=41, tag_tid=6, entry_tid=1' -a 'object_number=41, tag_tid=6, entry_tid=1' = 'object_number=41, tag_tid=6, entry_tid=1' 2024-04-23T21:23:27.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:27.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror test snap2 2024-04-23T21:23:27.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:27.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:27.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:27.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap2 2024-04-23T21:23:27.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:23:27.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:23:27.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:23:28.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/test@snap2 2024-04-23T21:23:28.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'test': 2024-04-23T21:23:28.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:23:28.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:23:28.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 3 2024-04-23T21:23:28.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10aff12cc611 2024-04-23T21:23:28.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10aff12cc611 2024-04-23T21:23:28.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:23:28.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:23:28.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:23:28.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:23:28.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:20:44 2024 2024-04-23T21:23:28.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:21:46 2024 2024-04-23T21:23:28.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:22:55 2024 2024-04-23T21:23:28.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:23:28.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10aff12cc611 2024-04-23T21:23:28.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:23:28.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:23:28.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:23:28.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:23:28.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:28.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n snap1-clean ']' 2024-04-23T21:23:28.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap1-clean 2024-04-23T21:23:28.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:28.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:28.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap1-clean 2024-04-23T21:23:28.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:23:28.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:23:28.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:23:28.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:23:28.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:23:29.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:23:29.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:23:29.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:23:29.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:23:29.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:23:29.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:30.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap2 2024-04-23T21:23:30.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:30.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:30.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap2 2024-04-23T21:23:30.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:23:30.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:23:30.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:23:30.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:23:30.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:23:30.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:23:30.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:23:30.651 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:23:30.651+0000 7fc3fe09e640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2024-04-23T21:23:30.777 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:23:30.778+0000 7f32a4d39640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 4 2024-04-23T21:23:31.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:23:31.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:23:31.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:23:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ clean_snap_name=snap2-clean 2024-04-23T21:23:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap2-clean 2024-04-23T21:23:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:23:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap2-clean 2024-04-23T21:23:31.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap2-clean --debug-rbd=20 --debug-journaler=20 2024-04-23T21:23:31.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:23:31.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror test 2024-04-23T21:23:31.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:23:31.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:31.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:31.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:23:31.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=9 2024-04-23T21:23:31.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:23:31.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2024-04-23T21:23:32.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 24647680 bytes 2024-04-23T21:23:34.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 81232896 bytes 2024-04-23T21:23:35.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 100544512 bytes 2024-04-23T21:23:35.824 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2024-04-23T21:23:35.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2024-04-23T21:23:35.825 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2024-04-23T21:23:35.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:23:35.827+0000 7fe84a38e640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 36190) UID: 1000 2024-04-23T21:23:36.826 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2024-04-23T21:23:36.826 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:23:36.826 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2024-04-23T21:23:36.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2024-04-23T21:23:36.827 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2024-04-23T21:23:36.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2024-04-23T21:23:36.829 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2024-04-23T21:23:36.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 136617984 bytes 2024-04-23T21:23:39.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 186482688 bytes 2024-04-23T21:23:40.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:600 IOs, 225622016 bytes 2024-04-23T21:23:40.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:23:40.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:23:40.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:23:40.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:40.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2024-04-23T21:23:40.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:40.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:40.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:40.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:23:40.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:23:40.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:23:40.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:23:40.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:23:41.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2024-04-23T21:23:41.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:41.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:41.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:41.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:23:41.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:23:41.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:23:41.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:23:41.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:23:41.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2024-04-23T21:23:41.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='test: 2024-04-23T21:23:41.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: global_id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:23:41.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+replaying 2024-04-23T21:23:41.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: replaying, {"bytes_per_second":0.0,"entries_behind_primary":7210,"entries_per_second":0.0,"non_primary_position":{"entry_tid":1,"object_number":41,"tag_tid":7},"primary_position":{"entry_tid":7210,"object_number":46,"tag_tid":8}} 2024-04-23T21:23:41.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: service: mirror.1 on smithi150 2024-04-23T21:23:41.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:23:38 2024-04-23T21:23:41.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: peer_sites: 2024-04-23T21:23:41.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: name: cluster2 2024-04-23T21:23:41.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+stopped 2024-04-23T21:23:41.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: local image is primary 2024-04-23T21:23:41.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:23:12' 2024-04-23T21:23:41.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ state: .*up+replaying' 2024-04-23T21:23:41.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: state: up+replaying 2024-04-23T21:23:41.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ description: .*' 2024-04-23T21:23:41.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: description: replaying, {"bytes_per_second":0.0,"entries_behind_primary":7210,"entries_per_second":0.0,"non_primary_position":{"entry_tid":1,"object_number":41,"tag_tid":7},"primary_position":{"entry_tid":7210,"object_number":46,"tag_tid":8}} 2024-04-23T21:23:41.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:23:41.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:23:41.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^up+' 2024-04-23T21:23:41.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:23:41.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep 'service: *mirror..* on ' 2024-04-23T21:23:41.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: service: mirror.1 on smithi150 2024-04-23T21:23:41.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update 2024-04-23T21:23:41.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-04-23T21:23:41.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:23:38' 2024-04-23T21:23:41.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2024-04-23 21:23:38' 2024-04-23T21:23:41.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:23:41.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:41.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:41.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:23:41.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'prev_last_update=2024-04-23 21:23:38' 2024-04-23T21:23:41.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:23:41.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:23:41.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-04-23T21:23:41.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='WARNINGWARNINGOK142184274mirror.1smithi15019.0.0-3212-g8254cb5afalseOK42224277mirror.3smithi15019.0.0-3212-g8254cb5afalseOK43434346mirror.2smithi15019.0.0-3212-g8254cb5atrueOK4367mirror.0smithi15019.0.0-3212-g8254cb5afalseWARNINGnot reporting statusteste56bd005-f8a9-476d-a8fc-713b4e8cd286up+replayingreplaying, {"bytes_per_second":0.0,"entries_behind_primary":7210,"entries_per_second":0.0,"non_primary_position":{"entry_tid":1,"object_number":41,"tag_tid":7},"primary_position":{"entry_tid":7210,"object_number":46,"tag_tid":8}}42184274mirror.1smithi1502024-04-23 21:23:38cluster267253cce-c621-4f62-a184-da88591999d7up+stoppedlocal image is primary2024-04-23 21:23:12' 2024-04-23T21:23:41.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update state 2024-04-23T21:23:41.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2024-04-23T21:23:41.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:23:38' 2024-04-23T21:23:41.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2024-04-23T21:23:41.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ state=up+replaying 2024-04-23T21:23:41.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:23:41.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep up+replaying 2024-04-23T21:23:41.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:23:41.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:41.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:23:41.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap3 2024-04-23T21:23:41.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap3 2024-04-23T21:23:41.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:23:41.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:23:41.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:23:41.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap3 2024-04-23T21:23:41.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap3 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:24:12.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror test 2024-04-23T21:24:12.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:12.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:12.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:12.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2024-04-23T21:24:12.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:12.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:12.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:12.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:24:12.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:24:12.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:24:12.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:24:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror test started 2024-04-23T21:24:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:24:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:24:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:24:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2024-04-23T21:24:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:24:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:24:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:24:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:24:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:24:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:24:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:24:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:24:13.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:24:13.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:24:13.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:24:13.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:24:13.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:24:13.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:24:13.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:24:13.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:24:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:24:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:24:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:24:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:24:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:24:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:24:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:24:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:24:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:24:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:24:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:24:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:24:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:24:13.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:24:13.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:24:13.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:24:13.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:24:13.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:24:13.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:24:13.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:24:13.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:24:13.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:24:13.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:24:13.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:24:13.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:24:13.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:13.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:13.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:13.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:24:13.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:24:13.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:24:13.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:24:13.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:24:13.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:24:13.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:24:13.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:24:13.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:24:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:24:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:24:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:24:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:24:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:24:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:24:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:24:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:24:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:24:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:24:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:24:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:24:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:24:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:24:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:24:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:24:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:24:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:24:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:24:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:24:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:24:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:24:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:24:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:24:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:24:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:24:13.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:13.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:13.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:13.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:24:14.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:24:14.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:24:14.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:24:14.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:24:14.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:24:14.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:24:14.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:24:14.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:24:14.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:24:14.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:24:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:24:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:24:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:24:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:24:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:24:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:24:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:24:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:24:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:24:14.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:24:14.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:24:14.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:14.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:24:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:24:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:24:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:24:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:24:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:24:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:24:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:24:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:24:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:24:14.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:24:14.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:24:14.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:24:14.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:24:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:24:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:24:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:24:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:24:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:24:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:24:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:24:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:24:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:24:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:24:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:24:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:24:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:24:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:24:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:24:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:24:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:24:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:24:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:24:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:24:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:24:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:24:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:24:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:24:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:24:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:24:14.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:14.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:14.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:14.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:24:14.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:24:14.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:24:14.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:24:14.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:24:14.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:24:14.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:24:14.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:14.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:24:14.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:24:14.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:24:14.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:24:14.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:24:14.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:24:14.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:24:14.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:24:14.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:24:14.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:24:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:24:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:24:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:24:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:24:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:24:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:24:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:24:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:24:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:24:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:24:14.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:24:14.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:24:14.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:24:14.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:24:14.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:24:14.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:24:14.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:14.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:14.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:14.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:24:14.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:24:14.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:24:14.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:14.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:14.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:14.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:24:14.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:24:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:24:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:24:14.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:24:14.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:14.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:14.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:24:14.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 13 2024-04-23T21:24:14.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 13 2024-04-23T21:24:14.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:24:14.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=53, tag_tid=9, entry_tid=1], [object_number=52, tag_tid=9, entry_tid=0], [object_number=55, tag_tid=8, entry_tid=14219], [object_number=54, tag_tid=8, entry_tid=14218]]], state=connected] 2024-04-23T21:24:14.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=55, tag_tid=8, entry_tid=13503], [object_number=54, tag_tid=8, entry_tid=13502], [object_number=53, tag_tid=8, entry_tid=13501], [object_number=52, tag_tid=8, entry_tid=13500]]], state=connected] 2024-04-23T21:24:14.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:14.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:14.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:24:14.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:14.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:14.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:14.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:24:14.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:14.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:14.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:14.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:24:14.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:24:14.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:24:14.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:24:14.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:14.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:24:14.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:14.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 13 2024-04-23T21:24:14.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 13 2024-04-23T21:24:14.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:24:14.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=53, tag_tid=9, entry_tid=1], [object_number=52, tag_tid=9, entry_tid=0], [object_number=55, tag_tid=8, entry_tid=14219], [object_number=54, tag_tid=8, entry_tid=14218]]], state=connected] 2024-04-23T21:24:14.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=55, tag_tid=8, entry_tid=13503], [object_number=54, tag_tid=8, entry_tid=13502], [object_number=53, tag_tid=8, entry_tid=13501], [object_number=52, tag_tid=8, entry_tid=13500]]], state=connected] 2024-04-23T21:24:14.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=55, tag_tid=8, entry_tid=13503' 2024-04-23T21:24:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=53, tag_tid=9, entry_tid=1' -a 'object_number=53, tag_tid=9, entry_tid=1' = 'object_number=55, tag_tid=8, entry_tid=13503' 2024-04-23T21:24:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=55, tag_tid=8, entry_tid=13503' '!=' '' 2024-04-23T21:24:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:24:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=55, tag_tid=8, entry_tid=13503' = '' 2024-04-23T21:24:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=55, tag_tid=8, entry_tid=13503' 2024-04-23T21:24:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:24:14.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:14.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=9 2024-04-23T21:24:14.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=55, tag_tid=8, entry_tid=13503' 2024-04-23T21:24:14.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:24:14.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:14.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=8 2024-04-23T21:24:14.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:14.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:24:14.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:14.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:24:14.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=55, tag_tid=8, entry_tid=13503' 2024-04-23T21:24:14.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:24:14.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:14.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=13503 2024-04-23T21:24:14.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 9 = 8 -a 1 -le 13503 2024-04-23T21:24:14.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:24:14.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:24:14.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:24:14.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:24:14.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:14.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:14.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:14.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:24:14.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:24:14.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:24:14.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:24:14.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:24:14.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:14.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:14.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:24:14.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:24:14.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:24:14.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:24:14.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:24:14.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:24:14.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:24:14.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:24:14.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:24:14.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:14.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:24:14.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:24:14.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:24:14.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:24:14.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:24:14.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:24:14.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:24:14.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:24:14.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:24:14.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:14.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:24:14.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:24:14.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:24:14.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:24:14.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:24:14.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:24:14.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:24:14.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:24:14.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:24:14.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:24:14.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:14.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:24:14.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:24:14.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:24:14.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:24:14.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:24:14.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:24:14.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:24:14.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:14.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:14.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:14.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:24:14.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:24:14.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:24:14.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:24:14.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:24:14.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:24:14.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:24:14.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:14.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:24:14.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:24:14.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:24:14.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:24:14.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:24:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:24:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:24:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:24:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:24:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:24:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:24:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:24:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:24:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:24:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:24:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:24:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:24:14.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:24:14.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:24:14.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:14.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:24:14.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:24:14.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:24:14.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:24:14.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:24:14.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:24:14.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:24:14.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:14.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:14.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:14.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:24:14.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:24:14.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:24:14.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:14.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:14.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:14.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:24:14.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:14.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:14.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:14.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:24:14.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:24:14.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:24:14.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:24:14.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:14.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:24:14.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:15.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 13 2024-04-23T21:24:15.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 13 2024-04-23T21:24:15.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:24:15.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=53, tag_tid=9, entry_tid=1], [object_number=52, tag_tid=9, entry_tid=0], [object_number=55, tag_tid=8, entry_tid=14219], [object_number=54, tag_tid=8, entry_tid=14218]]], state=connected] 2024-04-23T21:24:15.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=54, tag_tid=8, entry_tid=13714], [object_number=53, tag_tid=8, entry_tid=13713], [object_number=52, tag_tid=8, entry_tid=13712], [object_number=55, tag_tid=8, entry_tid=13711]]], state=connected] 2024-04-23T21:24:15.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:15.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:15.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:24:15.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:15.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:15.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:15.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:24:15.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:15.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:15.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:15.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:24:15.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:24:15.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:24:15.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:24:15.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:15.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:24:15.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:15.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 13 2024-04-23T21:24:15.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 13 2024-04-23T21:24:15.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:24:15.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=53, tag_tid=9, entry_tid=1], [object_number=52, tag_tid=9, entry_tid=0], [object_number=55, tag_tid=8, entry_tid=14219], [object_number=54, tag_tid=8, entry_tid=14218]]], state=connected] 2024-04-23T21:24:15.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=54, tag_tid=8, entry_tid=13714], [object_number=53, tag_tid=8, entry_tid=13713], [object_number=52, tag_tid=8, entry_tid=13712], [object_number=55, tag_tid=8, entry_tid=13711]]], state=connected] 2024-04-23T21:24:15.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:15.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=54, tag_tid=8, entry_tid=13714' 2024-04-23T21:24:15.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=53, tag_tid=9, entry_tid=1' -a 'object_number=53, tag_tid=9, entry_tid=1' = 'object_number=54, tag_tid=8, entry_tid=13714' 2024-04-23T21:24:15.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=54, tag_tid=8, entry_tid=13714' '!=' 'object_number=55, tag_tid=8, entry_tid=13503' 2024-04-23T21:24:15.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:24:15.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=54, tag_tid=8, entry_tid=13714' = 'object_number=55, tag_tid=8, entry_tid=13503' 2024-04-23T21:24:15.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=54, tag_tid=8, entry_tid=13714' 2024-04-23T21:24:15.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:15.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:24:15.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:15.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=9 2024-04-23T21:24:15.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=54, tag_tid=8, entry_tid=13714' 2024-04-23T21:24:15.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:24:15.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:15.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=8 2024-04-23T21:24:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:24:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:15.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:24:15.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=54, tag_tid=8, entry_tid=13714' 2024-04-23T21:24:15.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:24:15.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:15.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=13714 2024-04-23T21:24:15.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 9 = 8 -a 1 -le 13714 2024-04-23T21:24:15.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:24:15.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:24:15.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:24:15.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:24:15.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:15.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:15.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:15.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:24:15.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:24:15.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:24:15.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:24:15.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:24:15.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:15.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:15.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:24:15.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:24:15.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:24:15.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:24:15.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:24:15.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:24:15.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:24:15.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:24:15.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:24:15.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:15.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:24:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:24:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:24:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:24:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:24:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:24:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:24:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:24:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:24:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:24:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:24:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:24:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:24:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:24:15.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:24:15.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:24:15.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:24:15.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:24:15.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:24:15.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:15.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:24:15.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:24:15.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:24:15.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:24:15.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:24:15.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:24:15.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:24:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:15.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:24:15.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:24:15.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:24:15.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:24:15.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:24:15.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:24:15.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:24:15.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:15.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:24:15.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:24:15.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:24:15.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:24:15.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:24:15.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:24:15.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:24:15.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:24:15.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:24:15.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:15.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:24:15.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:24:15.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:24:15.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:24:15.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:24:15.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:24:15.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:24:15.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:24:15.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:24:15.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:24:15.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:15.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:24:15.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:24:15.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:24:15.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:24:15.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:24:15.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:24:15.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:24:15.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:15.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:15.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:15.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:24:15.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:24:15.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:24:15.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:15.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:15.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:15.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:24:15.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:15.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:15.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:15.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:24:15.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:24:15.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:24:15.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:24:15.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:15.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:24:15.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:15.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 13 2024-04-23T21:24:15.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 13 2024-04-23T21:24:15.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:24:15.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=53, tag_tid=9, entry_tid=1], [object_number=52, tag_tid=9, entry_tid=0], [object_number=55, tag_tid=8, entry_tid=14219], [object_number=54, tag_tid=8, entry_tid=14218]]], state=connected] 2024-04-23T21:24:15.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=54, tag_tid=8, entry_tid=13954], [object_number=53, tag_tid=8, entry_tid=13953], [object_number=52, tag_tid=8, entry_tid=13952], [object_number=55, tag_tid=8, entry_tid=13951]]], state=connected] 2024-04-23T21:24:15.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:15.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:15.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:24:15.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:15.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:15.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:15.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:24:15.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:15.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:15.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:15.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:24:15.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:24:15.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:24:15.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:24:15.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:15.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:24:15.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:15.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 13 2024-04-23T21:24:15.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 13 2024-04-23T21:24:15.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:24:15.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=53, tag_tid=9, entry_tid=1], [object_number=52, tag_tid=9, entry_tid=0], [object_number=55, tag_tid=8, entry_tid=14219], [object_number=54, tag_tid=8, entry_tid=14218]]], state=connected] 2024-04-23T21:24:15.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=54, tag_tid=8, entry_tid=13954], [object_number=53, tag_tid=8, entry_tid=13953], [object_number=52, tag_tid=8, entry_tid=13952], [object_number=55, tag_tid=8, entry_tid=13951]]], state=connected] 2024-04-23T21:24:15.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=54, tag_tid=8, entry_tid=13954' 2024-04-23T21:24:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=53, tag_tid=9, entry_tid=1' -a 'object_number=53, tag_tid=9, entry_tid=1' = 'object_number=54, tag_tid=8, entry_tid=13954' 2024-04-23T21:24:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=54, tag_tid=8, entry_tid=13954' '!=' 'object_number=54, tag_tid=8, entry_tid=13714' 2024-04-23T21:24:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:24:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=54, tag_tid=8, entry_tid=13954' = 'object_number=54, tag_tid=8, entry_tid=13714' 2024-04-23T21:24:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=54, tag_tid=8, entry_tid=13954' 2024-04-23T21:24:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:24:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=9 2024-04-23T21:24:15.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=54, tag_tid=8, entry_tid=13954' 2024-04-23T21:24:15.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:24:15.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:15.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=8 2024-04-23T21:24:15.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:15.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:24:15.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:15.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:24:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=54, tag_tid=8, entry_tid=13954' 2024-04-23T21:24:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:24:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:15.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=13954 2024-04-23T21:24:15.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 9 = 8 -a 1 -le 13954 2024-04-23T21:24:15.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:24:15.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:24:15.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:24:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:24:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:24:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:24:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:24:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:24:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:24:15.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:15.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:15.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:24:15.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:24:15.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:24:15.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:24:15.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:24:15.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:24:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:24:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:24:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:24:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:24:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:24:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:24:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:24:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:24:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:24:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:24:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:24:15.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:24:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:24:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:24:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:24:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:24:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:24:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:24:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:24:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:24:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:24:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:24:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:24:15.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:24:15.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:24:15.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:24:15.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:24:15.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:24:15.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:24:15.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:15.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:15.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:15.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:24:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:24:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:24:15.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:24:15.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:24:15.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:24:15.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:24:15.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:15.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:24:15.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:24:15.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:24:15.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:24:15.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:24:15.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:24:15.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:24:15.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:24:15.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:24:15.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:15.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:24:15.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:24:15.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:24:15.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:24:15.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:24:15.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:24:15.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:24:15.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:24:15.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:24:15.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:24:15.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:15.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:24:15.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:24:15.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:24:15.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:24:15.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:24:15.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:24:15.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:24:15.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:15.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:15.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:15.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:24:15.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:24:15.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:24:15.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:15.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:15.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:15.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:24:15.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:15.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:15.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:15.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:24:15.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:24:15.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:24:15.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:24:15.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:15.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:24:15.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:16.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 13 2024-04-23T21:24:16.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 13 2024-04-23T21:24:16.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:24:16.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=53, tag_tid=9, entry_tid=1], [object_number=52, tag_tid=9, entry_tid=0], [object_number=55, tag_tid=8, entry_tid=14219], [object_number=54, tag_tid=8, entry_tid=14218]]], state=connected] 2024-04-23T21:24:16.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=54, tag_tid=8, entry_tid=14198], [object_number=53, tag_tid=8, entry_tid=14197], [object_number=52, tag_tid=8, entry_tid=14196], [object_number=55, tag_tid=8, entry_tid=14195]]], state=connected] 2024-04-23T21:24:16.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:16.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:16.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:24:16.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:16.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:16.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:16.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:24:16.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:16.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:16.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:16.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:24:16.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:24:16.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:24:16.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:24:16.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:16.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:24:16.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:16.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 13 2024-04-23T21:24:16.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 13 2024-04-23T21:24:16.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:24:16.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=53, tag_tid=9, entry_tid=1], [object_number=52, tag_tid=9, entry_tid=0], [object_number=55, tag_tid=8, entry_tid=14219], [object_number=54, tag_tid=8, entry_tid=14218]]], state=connected] 2024-04-23T21:24:16.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=54, tag_tid=8, entry_tid=14198], [object_number=53, tag_tid=8, entry_tid=14197], [object_number=52, tag_tid=8, entry_tid=14196], [object_number=55, tag_tid=8, entry_tid=14195]]], state=connected] 2024-04-23T21:24:16.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:16.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=54, tag_tid=8, entry_tid=14198' 2024-04-23T21:24:16.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=53, tag_tid=9, entry_tid=1' -a 'object_number=53, tag_tid=9, entry_tid=1' = 'object_number=54, tag_tid=8, entry_tid=14198' 2024-04-23T21:24:16.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=54, tag_tid=8, entry_tid=14198' '!=' 'object_number=54, tag_tid=8, entry_tid=13954' 2024-04-23T21:24:16.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:24:16.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=54, tag_tid=8, entry_tid=14198' = 'object_number=54, tag_tid=8, entry_tid=13954' 2024-04-23T21:24:16.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=54, tag_tid=8, entry_tid=14198' 2024-04-23T21:24:16.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:16.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:24:16.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:16.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=9 2024-04-23T21:24:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=54, tag_tid=8, entry_tid=14198' 2024-04-23T21:24:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:24:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=8 2024-04-23T21:24:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:24:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:24:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=54, tag_tid=8, entry_tid=14198' 2024-04-23T21:24:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:24:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:16.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=14198 2024-04-23T21:24:16.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 9 = 8 -a 1 -le 14198 2024-04-23T21:24:16.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:24:16.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:24:16.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:24:16.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:24:16.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:16.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:16.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:16.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:24:16.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:24:16.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:24:16.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:24:16.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:24:16.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:16.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:16.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:24:16.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:24:16.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:24:16.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:24:16.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:24:16.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:24:16.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:24:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:24:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:24:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:24:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:24:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:24:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:24:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:24:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:24:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:24:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:24:16.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:24:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:24:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:24:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:24:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:24:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:24:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:24:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:24:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:24:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:24:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:24:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:16.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:24:16.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:24:16.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:24:16.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:24:16.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:24:16.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:24:16.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:24:16.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:16.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:16.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:16.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:24:16.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:24:16.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:24:16.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:24:16.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:24:16.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:24:16.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:24:16.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:16.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:24:16.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:24:16.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:24:16.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:24:16.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:24:16.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:24:16.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:24:16.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:24:16.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:24:16.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:16.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:24:16.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:24:16.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:24:16.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:24:16.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:24:16.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:24:16.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:24:16.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:24:16.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:24:16.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:24:16.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:16.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:24:16.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:24:16.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:24:16.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:24:16.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:24:16.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:24:16.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:24:16.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:16.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:16.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:16.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:24:16.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:24:16.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:24:16.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:16.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:16.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:16.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:24:16.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:16.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:16.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:16.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:24:16.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:24:16.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:24:16.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:24:16.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:16.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:24:16.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:16.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 13 2024-04-23T21:24:16.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 13 2024-04-23T21:24:16.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:24:16.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=53, tag_tid=9, entry_tid=1], [object_number=52, tag_tid=9, entry_tid=0], [object_number=55, tag_tid=8, entry_tid=14219], [object_number=54, tag_tid=8, entry_tid=14218]]], state=connected] 2024-04-23T21:24:16.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=55, tag_tid=8, entry_tid=14219], [object_number=54, tag_tid=8, entry_tid=14218], [object_number=53, tag_tid=8, entry_tid=14217], [object_number=52, tag_tid=8, entry_tid=14216]]], state=connected] 2024-04-23T21:24:16.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:16.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:16.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:24:16.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:24:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:24:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:24:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:24:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:24:16.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:16.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:24:16.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 13 2024-04-23T21:24:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 13 2024-04-23T21:24:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:24:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=53, tag_tid=9, entry_tid=1], [object_number=52, tag_tid=9, entry_tid=0], [object_number=55, tag_tid=8, entry_tid=14219], [object_number=54, tag_tid=8, entry_tid=14218]]], state=connected] 2024-04-23T21:24:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=55, tag_tid=8, entry_tid=14219], [object_number=54, tag_tid=8, entry_tid=14218], [object_number=53, tag_tid=8, entry_tid=14217], [object_number=52, tag_tid=8, entry_tid=14216]]], state=connected] 2024-04-23T21:24:16.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:16.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=55, tag_tid=8, entry_tid=14219' 2024-04-23T21:24:16.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=53, tag_tid=9, entry_tid=1' -a 'object_number=53, tag_tid=9, entry_tid=1' = 'object_number=55, tag_tid=8, entry_tid=14219' 2024-04-23T21:24:16.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=55, tag_tid=8, entry_tid=14219' '!=' 'object_number=54, tag_tid=8, entry_tid=14198' 2024-04-23T21:24:16.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:24:16.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=55, tag_tid=8, entry_tid=14219' = 'object_number=54, tag_tid=8, entry_tid=14198' 2024-04-23T21:24:16.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=55, tag_tid=8, entry_tid=14219' 2024-04-23T21:24:16.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:16.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:24:16.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:16.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=9 2024-04-23T21:24:16.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=55, tag_tid=8, entry_tid=14219' 2024-04-23T21:24:16.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:24:16.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:16.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=8 2024-04-23T21:24:16.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:16.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:24:16.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:16.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:24:16.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=55, tag_tid=8, entry_tid=14219' 2024-04-23T21:24:16.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:24:16.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:24:16.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=14219 2024-04-23T21:24:16.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 9 = 8 -a 1 -le 14219 2024-04-23T21:24:16.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:24:16.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:24:16.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:24:16.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:24:16.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:16.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:16.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:16.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:24:16.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:24:16.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:24:16.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:24:16.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:24:16.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:16.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:16.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:24:16.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:24:16.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:24:16.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:24:16.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:24:16.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:24:16.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:24:16.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:24:16.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:24:16.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:16.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:24:16.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:24:16.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:24:16.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:24:16.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:24:16.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:24:16.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:24:16.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:24:16.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:24:16.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:16.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:24:16.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:24:16.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:24:16.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:24:16.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:24:16.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:24:16.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:24:16.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:24:16.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:24:16.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:24:16.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:16.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:24:16.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:24:16.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:24:16.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:24:16.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:24:16.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:24:16.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:24:16.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:16.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:16.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:24:16.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:24:16.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:24:16.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:24:16.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:24:16.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:24:16.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:24:16.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:24:16.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:24:16.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:24:16.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:24:16.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:24:16.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:24:16.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:24:16.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:24:16.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:24:16.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:24:16.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:24:16.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:16.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:24:16.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:24:16.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:24:16.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:24:16.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:24:16.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:24:16.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:24:16.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:24:16.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:24:16.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:24:16.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:24:16.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:24:16.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:24:16.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:24:16.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:24:16.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:24:16.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:24:16.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:24:16.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:16.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:16.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:24:16.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:24:17.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:24:17.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:24:17.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:17.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:17.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:24:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:24:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:24:17.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:24:17.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:24:17.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:17.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:24:17.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:17.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 13 2024-04-23T21:24:17.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 13 2024-04-23T21:24:17.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:24:17.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=53, tag_tid=9, entry_tid=1], [object_number=52, tag_tid=9, entry_tid=0], [object_number=55, tag_tid=8, entry_tid=14219], [object_number=54, tag_tid=8, entry_tid=14218]]], state=connected] 2024-04-23T21:24:17.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=53, tag_tid=9, entry_tid=1], [object_number=52, tag_tid=9, entry_tid=0], [object_number=55, tag_tid=8, entry_tid=14219], [object_number=54, tag_tid=8, entry_tid=14218]]], state=connected] 2024-04-23T21:24:17.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:17.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:17.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:24:17.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:17.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:17.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:17.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:24:17.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:24:17.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:24:17.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:24:17.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:24:17.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:24:17.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:24:17.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:24:17.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:17.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:24:17.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:17.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 13 2024-04-23T21:24:17.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 13 2024-04-23T21:24:17.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:24:17.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=53, tag_tid=9, entry_tid=1], [object_number=52, tag_tid=9, entry_tid=0], [object_number=55, tag_tid=8, entry_tid=14219], [object_number=54, tag_tid=8, entry_tid=14218]]], state=connected] 2024-04-23T21:24:17.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=53, tag_tid=9, entry_tid=1], [object_number=52, tag_tid=9, entry_tid=0], [object_number=55, tag_tid=8, entry_tid=14219], [object_number=54, tag_tid=8, entry_tid=14218]]], state=connected] 2024-04-23T21:24:17.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:24:17.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:17.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=53, tag_tid=9, entry_tid=1' -a 'object_number=53, tag_tid=9, entry_tid=1' = 'object_number=53, tag_tid=9, entry_tid=1' 2024-04-23T21:24:17.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:24:17.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror test snap3 2024-04-23T21:24:17.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:17.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:17.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:17.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap3 2024-04-23T21:24:17.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:24:17.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:24:17.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:24:18.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/test@snap3 2024-04-23T21:24:18.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'test': 2024-04-23T21:24:18.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:24:18.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:24:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 5 2024-04-23T21:24:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10aff12cc611 2024-04-23T21:24:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10aff12cc611 2024-04-23T21:24:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:24:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:24:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:24:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:24:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:20:44 2024 2024-04-23T21:24:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:23:29 2024 2024-04-23T21:24:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:23:56 2024 2024-04-23T21:24:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:24:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10aff12cc611 2024-04-23T21:24:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:24:18.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:24:18.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:24:18.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:24:18.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:24:18.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n snap2-clean ']' 2024-04-23T21:24:18.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap2-clean 2024-04-23T21:24:18.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:18.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:18.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap2-clean 2024-04-23T21:24:18.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:24:18.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:24:18.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:24:18.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:24:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:24:18.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:24:18.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:24:18.662 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:24:18.663+0000 7fc3f9895640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:24:18.881 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:24:18.882+0000 7f32a9542640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 4 2024-04-23T21:24:19.083 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:24:19.084+0000 7f32a2534640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 1 2024-04-23T21:24:19.084 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:24:19.084+0000 7f32a4538640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 2 2024-04-23T21:24:19.084 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:24:19.085+0000 7f32a8540640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 2 2024-04-23T21:24:19.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:24:19.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:24:19.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:24:19.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:24:19.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap3 2024-04-23T21:24:19.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:19.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:19.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap3 2024-04-23T21:24:19.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:24:19.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:24:19.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:24:19.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:24:19.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:24:20.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:24:20.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:24:20.167 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:24:20.167+0000 7fc3fc89b640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:24:20.316 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:24:20.316+0000 7f9201bfb640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 1 2024-04-23T21:24:20.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:24:20.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:24:21.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:24:21.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:24:21.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ clean_snap_name=snap3-clean 2024-04-23T21:24:21.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap3-clean 2024-04-23T21:24:21.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:24:21.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:21.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:21.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap3-clean 2024-04-23T21:24:21.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap3-clean --debug-rbd=20 --debug-journaler=20 2024-04-23T21:24:21.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:24:21.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror test 2024-04-23T21:24:21.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:24:21.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:21.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:21.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:24:21.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=8 2024-04-23T21:24:21.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:24:21.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2024-04-23T21:24:24.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 58742784 bytes 2024-04-23T21:24:24.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 74465280 bytes 2024-04-23T21:24:26.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 100056064 bytes 2024-04-23T21:24:26.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 127470592 bytes 2024-04-23T21:24:28.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 167971840 bytes 2024-04-23T21:24:28.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:600 IOs, 177439744 bytes 2024-04-23T21:24:29.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:24:29.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:24:29.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:24:29.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:24:29.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2024-04-23T21:24:29.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:29.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:29.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:29.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:24:29.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:24:29.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:24:29.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:24:29.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:24:30.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2024-04-23T21:24:30.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:30.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:30.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:30.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:24:30.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:24:30.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:24:30.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:24:30.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:24:30.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2024-04-23T21:24:30.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='test: 2024-04-23T21:24:30.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: global_id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:24:30.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+replaying 2024-04-23T21:24:30.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: replaying, {"bytes_per_second":6107854.27,"entries_behind_primary":0,"entries_per_second":382.27,"non_primary_position":{"entry_tid":11621,"object_number":49,"tag_tid":8},"primary_position":{"entry_tid":7210,"object_number":46,"tag_tid":8}} 2024-04-23T21:24:30.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: service: mirror.1 on smithi150 2024-04-23T21:24:30.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:24:12 2024-04-23T21:24:30.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: peer_sites: 2024-04-23T21:24:30.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: name: cluster2 2024-04-23T21:24:30.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+stopped 2024-04-23T21:24:30.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: local image is primary 2024-04-23T21:24:30.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:24:12' 2024-04-23T21:24:30.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ state: .*up+replaying' 2024-04-23T21:24:30.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: state: up+replaying 2024-04-23T21:24:30.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ description: .*' 2024-04-23T21:24:30.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: description: replaying, {"bytes_per_second":6107854.27,"entries_behind_primary":0,"entries_per_second":382.27,"non_primary_position":{"entry_tid":11621,"object_number":49,"tag_tid":8},"primary_position":{"entry_tid":7210,"object_number":46,"tag_tid":8}} 2024-04-23T21:24:30.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:24:30.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:24:30.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^up+' 2024-04-23T21:24:30.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:24:30.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep 'service: *mirror..* on ' 2024-04-23T21:24:30.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: service: mirror.1 on smithi150 2024-04-23T21:24:30.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update 2024-04-23T21:24:30.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-04-23T21:24:30.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:24:12' 2024-04-23T21:24:30.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2024-04-23 21:24:12' 2024-04-23T21:24:30.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:30.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:30.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:30.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:24:30.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'prev_last_update=2024-04-23 21:24:12' 2024-04-23T21:24:30.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:24:30.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:24:30.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-04-23T21:24:30.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='OKOKOK142184274mirror.1smithi15019.0.0-3212-g8254cb5afalseOK42224277mirror.3smithi15019.0.0-3212-g8254cb5afalseOK43434346mirror.2smithi15019.0.0-3212-g8254cb5atrueOK43674369mirror.0smithi15019.0.0-3212-g8254cb5afalseOKteste56bd005-f8a9-476d-a8fc-713b4e8cd286up+replayingreplaying, {"bytes_per_second":6107854.27,"entries_behind_primary":0,"entries_per_second":382.27,"non_primary_position":{"entry_tid":11621,"object_number":49,"tag_tid":8},"primary_position":{"entry_tid":7210,"object_number":46,"tag_tid":8}}42184274mirror.1smithi1502024-04-23 21:24:12cluster267253cce-c621-4f62-a184-da88591999d7up+stoppedlocal image is primary2024-04-23 21:24:12' 2024-04-23T21:24:30.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update state 2024-04-23T21:24:30.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2024-04-23T21:24:30.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:24:12' 2024-04-23T21:24:30.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2024-04-23T21:24:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ state=up+replaying 2024-04-23T21:24:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:24:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep up+replaying 2024-04-23T21:24:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:24:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:24:30.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:24:30.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap4 2024-04-23T21:24:30.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap4 2024-04-23T21:24:30.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:24:30.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:30.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:30.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap4 2024-04-23T21:24:30.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap4 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:24:52.721 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.751241453231944) > weight (0.25) 2024-04-23T21:24:52.721 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8197275040191326) > weight (0.25) 2024-04-23T21:24:52.721 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3717162252823757) > weight (0.25) 2024-04-23T21:24:52.721 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2024-04-23T21:24:52.722 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2024-04-23T21:24:52.722 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2024-04-23T21:24:52.723 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:24:52.724+0000 7f205ad30640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 35522) UID: 1000 2024-04-23T21:24:53.830 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6467110001274582) > weight (0.25) 2024-04-23T21:24:53.830 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6672557834512521) > weight (0.25) 2024-04-23T21:24:53.830 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.32122366842176164) > weight (0.25) 2024-04-23T21:24:53.830 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6959734689536721) > weight (0.25) 2024-04-23T21:24:53.830 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2024-04-23T21:24:56.722 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2024-04-23T21:24:56.723 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:24:56.723 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2024-04-23T21:24:56.723 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2024-04-23T21:24:56.723 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2024-04-23T21:24:56.726 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2024-04-23T21:24:56.726 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2024-04-23T21:24:58.885 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:24:58.885+0000 7f83e9354640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 1 2024-04-23T21:24:59.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror test 2024-04-23T21:24:59.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:59.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:59.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:59.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2024-04-23T21:24:59.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:24:59.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:24:59.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:24:59.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:24:59.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:24:59.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:24:59.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:25:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror test started 2024-04-23T21:25:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:25:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:25:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:25:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2024-04-23T21:25:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:25:00.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:25:00.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:25:00.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:00.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:25:00.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:25:00.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:25:00.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:25:00.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:25:00.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:25:00.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:00.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:25:00.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:25:00.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:25:00.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:00.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:25:00.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:25:00.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:25:00.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:00.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:00.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:00.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:00.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:25:00.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:25:00.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:25:00.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:25:00.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:25:00.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:25:00.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:25:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:25:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:25:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:25:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:25:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:25:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:25:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:25:00.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:00.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:25:00.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:25:00.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:25:00.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:25:00.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:25:00.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:00.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:00.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:00.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:25:00.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:25:00.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:25:00.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:25:00.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:00.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:25:00.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:25:00.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:25:00.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:00.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:25:00.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:25:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:25:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:25:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:25:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:25:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:25:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:25:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:25:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:25:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:25:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:25:00.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:25:00.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:25:00.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:25:00.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:25:00.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:25:00.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:00.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:25:00.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:25:00.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:25:00.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:25:00.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:25:01.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:01.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:01.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:01.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:25:01.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:25:01.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:25:01.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:25:01.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:25:01.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:25:01.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:25:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:25:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:25:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:25:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:25:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:25:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:25:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:25:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:25:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:01.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:25:01.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:25:01.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:25:01.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:25:01.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:25:01.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:25:01.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:01.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:01.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:01.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:25:01.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:25:01.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:25:01.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:25:01.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:25:01.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:01.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:01.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:25:01.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:25:01.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:25:01.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:25:01.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:25:01.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:25:01.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:01.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:25:01.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:25:01.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:01.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:01.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:25:01.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:25:01.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:01.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:01.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:01.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:25:01.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:01.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:01.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:01.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:25:01.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:01.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:25:01.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:25:01.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:01.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:01.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:25:01.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:25:01.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:01.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:01.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:01.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:01.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:01.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:01.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:01.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:25:01.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:25:01.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:25:01.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:01.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:01.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:01.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:25:01.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:25:01.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:25:01.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:25:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:25:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:25:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:01.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:25:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:25:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:25:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:25:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:01.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:25:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:25:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:25:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:25:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:01.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:25:01.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:25:01.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:25:01.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:01.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:01.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:01.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:25:01.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:01.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:25:01.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:01.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:01.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:01.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:25:01.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:01.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:01.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:01.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:25:01.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:25:01.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:25:01.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:25:01.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:01.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:25:01.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:01.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 16 2024-04-23T21:25:01.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 16 2024-04-23T21:25:01.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:25:01.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=65, tag_tid=12, entry_tid=1], [object_number=64, tag_tid=12, entry_tid=0], [object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442]]], state=connected] 2024-04-23T21:25:01.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442], [object_number=65, tag_tid=11, entry_tid=12441], [object_number=64, tag_tid=11, entry_tid=12440]]], state=connected] 2024-04-23T21:25:01.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:01.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=65, tag_tid=12, entry_tid=1' 2024-04-23T21:25:01.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:25:01.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:01.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:01.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:01.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:25:01.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:01.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:01.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:01.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:25:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:25:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:25:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:25:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:25:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:01.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 16 2024-04-23T21:25:01.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 16 2024-04-23T21:25:01.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:25:01.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=65, tag_tid=12, entry_tid=1], [object_number=64, tag_tid=12, entry_tid=0], [object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442]]], state=connected] 2024-04-23T21:25:01.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442], [object_number=65, tag_tid=11, entry_tid=12441], [object_number=64, tag_tid=11, entry_tid=12440]]], state=connected] 2024-04-23T21:25:01.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:01.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=67, tag_tid=11, entry_tid=12443' 2024-04-23T21:25:01.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=65, tag_tid=12, entry_tid=1' -a 'object_number=65, tag_tid=12, entry_tid=1' = 'object_number=67, tag_tid=11, entry_tid=12443' 2024-04-23T21:25:01.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=67, tag_tid=11, entry_tid=12443' '!=' '' 2024-04-23T21:25:01.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:25:01.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=67, tag_tid=11, entry_tid=12443' = '' 2024-04-23T21:25:01.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=67, tag_tid=11, entry_tid=12443' 2024-04-23T21:25:01.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=65, tag_tid=12, entry_tid=1' 2024-04-23T21:25:01.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:25:01.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:25:01.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=12 2024-04-23T21:25:01.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=67, tag_tid=11, entry_tid=12443' 2024-04-23T21:25:01.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:25:01.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:25:01.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=11 2024-04-23T21:25:01.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=65, tag_tid=12, entry_tid=1' 2024-04-23T21:25:01.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:25:01.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:25:01.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:25:01.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=67, tag_tid=11, entry_tid=12443' 2024-04-23T21:25:01.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:25:01.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:25:01.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=12443 2024-04-23T21:25:01.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 12 = 11 -a 1 -le 12443 2024-04-23T21:25:01.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:25:01.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:25:01.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:25:02.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:25:02.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:02.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:02.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:02.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:25:02.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:25:02.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:25:02.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:25:02.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:25:02.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:02.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:02.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:25:02.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:25:02.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:25:02.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:25:02.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:25:02.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:25:02.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:02.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:25:02.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:25:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:25:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:25:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:25:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:25:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:25:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:25:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:25:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:25:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:25:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:25:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:25:02.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:02.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:02.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:02.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:25:02.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:25:02.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:25:02.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:25:02.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:02.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:25:02.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:25:02.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:02.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:02.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:25:02.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:25:02.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:02.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:02.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:02.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:25:02.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:02.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:02.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:02.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:25:02.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:02.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:25:02.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:25:02.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:02.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:02.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:25:02.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:25:02.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:02.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:02.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:02.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:02.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:02.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:02.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:02.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:25:02.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:25:02.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:25:02.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:02.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:02.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:02.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:25:02.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:02.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:25:02.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:02.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:02.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:02.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:25:02.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:02.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:02.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:02.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:25:02.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:25:02.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:25:02.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:25:02.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:02.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:25:02.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:02.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 16 2024-04-23T21:25:02.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 16 2024-04-23T21:25:02.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:25:02.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=65, tag_tid=12, entry_tid=1], [object_number=64, tag_tid=12, entry_tid=0], [object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442]]], state=connected] 2024-04-23T21:25:02.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442], [object_number=65, tag_tid=11, entry_tid=12441], [object_number=64, tag_tid=11, entry_tid=12440]]], state=connected] 2024-04-23T21:25:02.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:02.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=65, tag_tid=12, entry_tid=1' 2024-04-23T21:25:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:25:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:25:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:25:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:25:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:25:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:25:02.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:02.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:25:02.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:02.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 16 2024-04-23T21:25:02.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 16 2024-04-23T21:25:02.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:25:02.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=65, tag_tid=12, entry_tid=1], [object_number=64, tag_tid=12, entry_tid=0], [object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442]]], state=connected] 2024-04-23T21:25:02.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442], [object_number=65, tag_tid=11, entry_tid=12441], [object_number=64, tag_tid=11, entry_tid=12440]]], state=connected] 2024-04-23T21:25:02.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:02.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=67, tag_tid=11, entry_tid=12443' 2024-04-23T21:25:02.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=65, tag_tid=12, entry_tid=1' -a 'object_number=65, tag_tid=12, entry_tid=1' = 'object_number=67, tag_tid=11, entry_tid=12443' 2024-04-23T21:25:02.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=67, tag_tid=11, entry_tid=12443' '!=' 'object_number=67, tag_tid=11, entry_tid=12443' 2024-04-23T21:25:02.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:25:02.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:25:02.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:25:02.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:02.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:02.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:02.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:25:02.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:25:02.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:25:02.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:25:02.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:25:02.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:02.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:02.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:25:02.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:25:02.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:25:02.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:25:02.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:25:02.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:25:02.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:02.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:25:02.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:25:02.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:02.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:02.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:25:02.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:25:02.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:02.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:02.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:02.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:25:02.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:02.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:02.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:02.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:25:02.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:02.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:25:02.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:25:02.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:02.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:25:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:25:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:25:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:25:02.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:25:02.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:02.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:02.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:02.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:25:02.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:25:02.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:25:02.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:25:02.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:02.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:25:02.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:25:02.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:02.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:02.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:25:02.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:25:02.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:02.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:02.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:02.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:25:02.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:02.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:02.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:02.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:25:02.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:02.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:25:02.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:25:02.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:02.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:02.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:25:02.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:25:02.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:02.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:02.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:02.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:02.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:02.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:02.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:02.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:25:02.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:25:02.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:25:03.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:03.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:03.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:03.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:25:03.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:03.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:25:03.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:25:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:25:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:25:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:25:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:25:03.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:03.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:25:03.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:03.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 16 2024-04-23T21:25:03.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 16 2024-04-23T21:25:03.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:25:03.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=65, tag_tid=12, entry_tid=1], [object_number=64, tag_tid=12, entry_tid=0], [object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442]]], state=connected] 2024-04-23T21:25:03.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442], [object_number=65, tag_tid=11, entry_tid=12441], [object_number=64, tag_tid=11, entry_tid=12440]]], state=connected] 2024-04-23T21:25:03.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:03.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=65, tag_tid=12, entry_tid=1' 2024-04-23T21:25:03.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:25:03.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:03.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:03.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:03.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:25:03.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:03.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:03.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:03.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:25:03.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:25:03.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:25:03.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:25:03.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:03.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:25:03.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:03.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 16 2024-04-23T21:25:03.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 16 2024-04-23T21:25:03.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:25:03.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=65, tag_tid=12, entry_tid=1], [object_number=64, tag_tid=12, entry_tid=0], [object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442]]], state=connected] 2024-04-23T21:25:03.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442], [object_number=65, tag_tid=11, entry_tid=12441], [object_number=64, tag_tid=11, entry_tid=12440]]], state=connected] 2024-04-23T21:25:03.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:03.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=67, tag_tid=11, entry_tid=12443' 2024-04-23T21:25:03.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=65, tag_tid=12, entry_tid=1' -a 'object_number=65, tag_tid=12, entry_tid=1' = 'object_number=67, tag_tid=11, entry_tid=12443' 2024-04-23T21:25:03.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=67, tag_tid=11, entry_tid=12443' '!=' 'object_number=67, tag_tid=11, entry_tid=12443' 2024-04-23T21:25:03.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:25:03.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:25:03.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:25:03.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:03.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:03.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:03.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:25:03.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:25:03.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:25:03.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:25:03.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:25:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:25:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:25:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:25:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:25:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:25:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:25:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:25:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:25:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:25:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:25:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:25:03.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:03.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:03.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:03.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:25:03.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:03.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:25:03.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:25:03.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:03.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:03.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:25:03.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:25:03.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:03.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:03.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:03.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:03.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:03.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:03.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:03.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:25:03.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:25:03.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:25:04.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:04.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:04.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:04.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:25:04.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:25:04.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:25:04.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:25:04.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:04.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:25:04.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:25:04.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:04.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:04.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:25:04.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:25:04.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:04.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:04.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:04.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:25:04.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:04.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:04.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:04.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:25:04.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:04.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:25:04.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:25:04.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:25:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:25:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:25:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:25:04.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:25:04.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:04.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:04.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:04.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:25:04.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:04.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:25:04.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:04.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:04.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:04.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:25:04.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:04.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:04.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:04.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:25:04.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:25:04.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:25:04.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:25:04.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:04.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:25:04.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:04.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 16 2024-04-23T21:25:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 16 2024-04-23T21:25:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:25:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=65, tag_tid=12, entry_tid=1], [object_number=64, tag_tid=12, entry_tid=0], [object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442]]], state=connected] 2024-04-23T21:25:04.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442], [object_number=65, tag_tid=11, entry_tid=12441], [object_number=64, tag_tid=11, entry_tid=12440]]], state=connected] 2024-04-23T21:25:04.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:04.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=65, tag_tid=12, entry_tid=1' 2024-04-23T21:25:04.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:25:04.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:04.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:04.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:04.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:25:04.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:04.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:04.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:25:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:25:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:25:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:25:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:25:04.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:04.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 16 2024-04-23T21:25:04.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 16 2024-04-23T21:25:04.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:25:04.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=65, tag_tid=12, entry_tid=1], [object_number=64, tag_tid=12, entry_tid=0], [object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442]]], state=connected] 2024-04-23T21:25:04.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442], [object_number=65, tag_tid=11, entry_tid=12441], [object_number=64, tag_tid=11, entry_tid=12440]]], state=connected] 2024-04-23T21:25:04.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:04.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=67, tag_tid=11, entry_tid=12443' 2024-04-23T21:25:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=65, tag_tid=12, entry_tid=1' -a 'object_number=65, tag_tid=12, entry_tid=1' = 'object_number=67, tag_tid=11, entry_tid=12443' 2024-04-23T21:25:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=67, tag_tid=11, entry_tid=12443' '!=' 'object_number=67, tag_tid=11, entry_tid=12443' 2024-04-23T21:25:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:25:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:25:05.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:25:05.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:05.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:05.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:05.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:25:05.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:25:05.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:25:05.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:25:05.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:25:05.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:05.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:05.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:25:05.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:25:05.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:25:05.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:25:05.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:25:05.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:25:05.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:05.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:25:05.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:25:05.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:05.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:25:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:25:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:25:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:05.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:25:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:25:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:25:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:25:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:25:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:25:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:25:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:25:05.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:05.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:05.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:05.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:25:06.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:25:06.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:25:06.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:25:06.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:06.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:25:06.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:25:06.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:06.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:06.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:25:06.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:25:06.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:06.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:06.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:06.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:25:06.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:06.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:06.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:06.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:25:06.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:06.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:25:06.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:25:06.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:06.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:06.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:25:06.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:25:06.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:06.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:06.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:06.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:06.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:06.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:06.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:06.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:25:06.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:25:06.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:25:06.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:06.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:06.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:06.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:25:06.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:06.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:25:06.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:06.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:06.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:06.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:25:06.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:06.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:06.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:06.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:25:06.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:25:06.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:25:06.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:25:06.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:06.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:25:06.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:06.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 16 2024-04-23T21:25:06.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 16 2024-04-23T21:25:06.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:25:06.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=65, tag_tid=12, entry_tid=1], [object_number=64, tag_tid=12, entry_tid=0], [object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442]]], state=connected] 2024-04-23T21:25:06.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=65, tag_tid=12, entry_tid=1], [object_number=64, tag_tid=12, entry_tid=0], [object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442]]], state=connected] 2024-04-23T21:25:06.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:06.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=65, tag_tid=12, entry_tid=1' 2024-04-23T21:25:06.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:25:06.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:06.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:06.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:06.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:25:06.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:06.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:06.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:06.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:25:06.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:25:06.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:25:06.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:25:06.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:06.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:25:06.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:06.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 16 2024-04-23T21:25:06.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 16 2024-04-23T21:25:06.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:25:06.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=65, tag_tid=12, entry_tid=1], [object_number=64, tag_tid=12, entry_tid=0], [object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442]]], state=connected] 2024-04-23T21:25:06.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=65, tag_tid=12, entry_tid=1], [object_number=64, tag_tid=12, entry_tid=0], [object_number=67, tag_tid=11, entry_tid=12443], [object_number=66, tag_tid=11, entry_tid=12442]]], state=connected] 2024-04-23T21:25:06.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:06.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=65, tag_tid=12, entry_tid=1' 2024-04-23T21:25:06.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=65, tag_tid=12, entry_tid=1' -a 'object_number=65, tag_tid=12, entry_tid=1' = 'object_number=65, tag_tid=12, entry_tid=1' 2024-04-23T21:25:06.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:06.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror test snap4 2024-04-23T21:25:06.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:06.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:06.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:06.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap4 2024-04-23T21:25:06.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:25:06.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:25:06.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:25:07.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/test@snap4 2024-04-23T21:25:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'test': 2024-04-23T21:25:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:25:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:25:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 7 2024-04-23T21:25:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10aff12cc611 2024-04-23T21:25:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10aff12cc611 2024-04-23T21:25:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:25:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:25:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:25:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:25:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:20:44 2024 2024-04-23T21:25:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:23:29 2024 2024-04-23T21:25:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:24:57 2024 2024-04-23T21:25:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:25:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10aff12cc611 2024-04-23T21:25:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:25:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:25:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:25:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:25:07.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:07.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n snap3-clean ']' 2024-04-23T21:25:07.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap3-clean 2024-04-23T21:25:07.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:07.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:07.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap3-clean 2024-04-23T21:25:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:25:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:25:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:25:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:25:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:25:07.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:25:07.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:25:08.200 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:25:08.201+0000 7fc3f8893640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2024-04-23T21:25:08.202 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:25:08.202+0000 7fc3fc89b640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2024-04-23T21:25:08.412 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:25:08.412+0000 7f32a4538640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 5 2024-04-23T21:25:08.618 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:25:08.618+0000 7fc3fd09c640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:25:08.619 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:25:08.620+0000 7fc3f688f640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 6 2024-04-23T21:25:08.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:25:08.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:25:08.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:25:09.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:09.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap4 2024-04-23T21:25:09.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:09.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:09.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap4 2024-04-23T21:25:09.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:25:09.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:25:09.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:25:09.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:25:09.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:25:09.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:25:09.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:25:10.212 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:25:10.213+0000 7fc3fd09c640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:25:10.214 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:25:10.215+0000 7fc3f688f640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 5 2024-04-23T21:25:10.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:25:10.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:25:10.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:25:10.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:10.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ clean_snap_name=snap4-clean 2024-04-23T21:25:10.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap4-clean 2024-04-23T21:25:10.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:25:10.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:10.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:10.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap4-clean 2024-04-23T21:25:10.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap4-clean --debug-rbd=20 --debug-journaler=20 2024-04-23T21:25:10.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:25:10.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror test 2024-04-23T21:25:10.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:25:10.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:10.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:10.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:25:10.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=14 2024-04-23T21:25:10.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:25:10.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2024-04-23T21:25:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 59446272 bytes 2024-04-23T21:25:14.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 77346816 bytes 2024-04-23T21:25:15.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 83704832 bytes 2024-04-23T21:25:15.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 97703936 bytes 2024-04-23T21:25:17.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 161245184 bytes 2024-04-23T21:25:20.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:600 IOs, 215095296 bytes 2024-04-23T21:25:21.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:700 IOs, 239325184 bytes 2024-04-23T21:25:22.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:800 IOs, 276632576 bytes 2024-04-23T21:25:24.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:900 IOs, 329461760 bytes 2024-04-23T21:25:24.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:25:24.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:25:24.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:25:24.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:24.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2024-04-23T21:25:24.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:24.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:24.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:24.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:25:24.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:25:24.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:25:24.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:25:24.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:25:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2024-04-23T21:25:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:25:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:25:25.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:25:25.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:25:25.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:25:25.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2024-04-23T21:25:25.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='test: 2024-04-23T21:25:25.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: global_id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:25:25.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+replaying 2024-04-23T21:25:25.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: replaying, {"bytes_per_second":4546150.63,"entries_behind_primary":0,"entries_per_second":287.93,"non_primary_position":{"entry_tid":1,"object_number":65,"tag_tid":12},"primary_position":{"entry_tid":1,"object_number":65,"tag_tid":12}} 2024-04-23T21:25:25.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: service: mirror.1 on smithi150 2024-04-23T21:25:25.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:25:12 2024-04-23T21:25:25.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: peer_sites: 2024-04-23T21:25:25.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: name: cluster2 2024-04-23T21:25:25.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+stopped 2024-04-23T21:25:25.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: local image is primary 2024-04-23T21:25:25.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:25:12' 2024-04-23T21:25:25.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ state: .*up+replaying' 2024-04-23T21:25:25.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: state: up+replaying 2024-04-23T21:25:25.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ description: .*' 2024-04-23T21:25:25.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: description: replaying, {"bytes_per_second":4546150.63,"entries_behind_primary":0,"entries_per_second":287.93,"non_primary_position":{"entry_tid":1,"object_number":65,"tag_tid":12},"primary_position":{"entry_tid":1,"object_number":65,"tag_tid":12}} 2024-04-23T21:25:25.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:25:25.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:25:25.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^up+' 2024-04-23T21:25:25.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:25:25.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep 'service: *mirror..* on ' 2024-04-23T21:25:25.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: service: mirror.1 on smithi150 2024-04-23T21:25:25.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update 2024-04-23T21:25:25.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-04-23T21:25:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:25:12' 2024-04-23T21:25:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2024-04-23 21:25:12' 2024-04-23T21:25:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:25:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'prev_last_update=2024-04-23 21:25:12' 2024-04-23T21:25:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:25:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:25:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-04-23T21:25:26.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='OKOKOK142184274mirror.1smithi15019.0.0-3212-g8254cb5afalseOK42224277mirror.3smithi15019.0.0-3212-g8254cb5afalseOK43434346mirror.2smithi15019.0.0-3212-g8254cb5atrueOK43674369mirror.0smithi15019.0.0-3212-g8254cb5afalseOKteste56bd005-f8a9-476d-a8fc-713b4e8cd286up+replayingreplaying, {"bytes_per_second":4546150.63,"entries_behind_primary":0,"entries_per_second":287.93,"non_primary_position":{"entry_tid":1,"object_number":65,"tag_tid":12},"primary_position":{"entry_tid":1,"object_number":65,"tag_tid":12}}42184274mirror.1smithi1502024-04-23 21:25:12cluster267253cce-c621-4f62-a184-da88591999d7up+stoppedlocal image is primary2024-04-23 21:25:12' 2024-04-23T21:25:26.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update state 2024-04-23T21:25:26.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2024-04-23T21:25:26.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:25:12' 2024-04-23T21:25:26.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2024-04-23T21:25:26.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ state=up+replaying 2024-04-23T21:25:26.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:25:26.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep up+replaying 2024-04-23T21:25:26.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:25:26.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:26.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:26.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap5 2024-04-23T21:25:26.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap5 2024-04-23T21:25:26.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:25:26.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:26.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:26.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap5 2024-04-23T21:25:26.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap5 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:25:56.070 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:25:56.069+0000 7fe967b5e640 -1 bdev(0x55a5abf68e00 /var/lib/ceph/osd/cluster2-2/block) aio_submit retries 3 2024-04-23T21:25:56.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror test 2024-04-23T21:25:56.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:56.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:56.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:56.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2024-04-23T21:25:56.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:56.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:56.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:56.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:25:56.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:25:56.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:25:56.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:25:57.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror test started 2024-04-23T21:25:57.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:57.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:57.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:57.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:25:57.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:25:57.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:25:57.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2024-04-23T21:25:57.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:25:57.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:25:57.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:25:57.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:57.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:25:57.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:25:57.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:25:57.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:25:57.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:25:57.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:25:57.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:57.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:25:57.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:25:57.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:25:57.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:57.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:25:57.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:25:57.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:25:57.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:57.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:57.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:57.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:57.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:25:57.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:25:57.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:25:57.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:25:57.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:25:57.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:25:57.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:25:57.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:25:57.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:25:57.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:25:57.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:25:57.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:25:57.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:25:57.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:25:57.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:57.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:25:57.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:25:57.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:25:57.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:25:57.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:25:57.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:57.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:57.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:57.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:25:58.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:25:58.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:25:58.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:25:58.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:58.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:25:58.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:25:58.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:25:58.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:58.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:25:58.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:25:58.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:25:58.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:58.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:58.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:58.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:58.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:25:58.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:25:58.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:25:58.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:25:58.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:25:58.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:25:58.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:25:58.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:25:58.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:25:58.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:25:58.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:25:58.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:25:58.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:25:58.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:25:58.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:58.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:25:58.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:25:58.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:25:58.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:25:58.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:25:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:58.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:25:58.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:25:58.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:25:58.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:25:58.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:25:58.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:25:58.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:25:58.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:25:58.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:58.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:25:58.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:25:58.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:25:58.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:58.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:58.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:25:58.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:25:58.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:25:58.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:25:58.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:58.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:58.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:25:58.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:25:58.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:25:58.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:25:58.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:25:58.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:25:58.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:25:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:25:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:25:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:25:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:25:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:25:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:25:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:25:58.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:25:58.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:25:58.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:25:58.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:58.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:25:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:25:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:25:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:25:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:25:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:58.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:25:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:25:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:25:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:25:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:25:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:58.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:58.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:25:58.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:25:58.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:25:58.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:58.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:58.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:58.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:25:58.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:25:58.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:25:58.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:25:58.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:58.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:25:58.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:25:58.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:58.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:58.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:25:58.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:25:58.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:58.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:58.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:58.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:25:58.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:58.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:58.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:58.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:25:58.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:58.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:25:58.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:25:58.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:58.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:58.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:25:58.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:25:58.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:58.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:58.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:58.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:58.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:58.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:58.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:58.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:25:58.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:25:58.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:25:58.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:58.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:58.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:58.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:25:58.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:58.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:25:58.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:58.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:58.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:58.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:25:58.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:58.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:58.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:58.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:25:58.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:25:58.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:25:58.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:25:58.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:58.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:25:58.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:25:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:25:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:25:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:25:58.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=82, tag_tid=14, entry_tid=15766], [object_number=81, tag_tid=14, entry_tid=15765], [object_number=80, tag_tid=14, entry_tid=15764], [object_number=83, tag_tid=14, entry_tid=15763]]], state=connected] 2024-04-23T21:25:58.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:25:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:25:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:25:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:58.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:25:58.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:25:58.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:25:58.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:25:58.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:58.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:25:58.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:58.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:25:58.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:25:58.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:25:58.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:25:58.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=82, tag_tid=14, entry_tid=15766], [object_number=81, tag_tid=14, entry_tid=15765], [object_number=80, tag_tid=14, entry_tid=15764], [object_number=83, tag_tid=14, entry_tid=15763]]], state=connected] 2024-04-23T21:25:58.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:58.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=82, tag_tid=14, entry_tid=15766' 2024-04-23T21:25:58.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=82, tag_tid=14, entry_tid=15766' 2024-04-23T21:25:58.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=82, tag_tid=14, entry_tid=15766' '!=' '' 2024-04-23T21:25:58.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:25:58.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=82, tag_tid=14, entry_tid=15766' = '' 2024-04-23T21:25:58.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=82, tag_tid=14, entry_tid=15766' 2024-04-23T21:25:58.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:25:58.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:25:58.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:25:58.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:25:58.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=82, tag_tid=14, entry_tid=15766' 2024-04-23T21:25:58.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:25:58.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:25:58.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:25:58.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:25:58.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:25:58.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:25:58.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:25:58.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=82, tag_tid=14, entry_tid=15766' 2024-04-23T21:25:58.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:25:58.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:25:58.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=15766 2024-04-23T21:25:58.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 15766 2024-04-23T21:25:58.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:25:58.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:25:58.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:25:58.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:25:58.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:58.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:58.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:58.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:25:58.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:25:58.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:25:58.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:25:58.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:25:58.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:58.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:58.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:25:58.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:25:58.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:25:58.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:25:58.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:25:58.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:25:58.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:58.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:25:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:25:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:25:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:25:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:25:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:58.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:25:58.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:58.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:25:58.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:25:58.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:58.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:58.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:25:58.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:25:58.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:58.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:58.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:58.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:58.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:58.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:58.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:58.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:25:58.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:25:58.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:25:58.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:58.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:58.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:58.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:25:58.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:25:58.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:25:58.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:25:58.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:58.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:25:58.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:25:58.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:58.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:58.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:25:58.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:25:58.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:58.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:58.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:58.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:25:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:25:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:25:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:25:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:25:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:25:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:58.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:58.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:58.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:58.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:25:58.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:25:58.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:25:58.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:58.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:58.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:58.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:25:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:25:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:25:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:25:59.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:25:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:25:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:25:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:25:59.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:59.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:25:59.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:25:59.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:25:59.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:25:59.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=80, tag_tid=14, entry_tid=15968], [object_number=83, tag_tid=14, entry_tid=15967], [object_number=82, tag_tid=14, entry_tid=15966], [object_number=81, tag_tid=14, entry_tid=15965]]], state=connected] 2024-04-23T21:25:59.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:25:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:25:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:59.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:25:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:25:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:25:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:25:59.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:25:59.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:59.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:25:59.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:59.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:25:59.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:25:59.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:25:59.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:25:59.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=80, tag_tid=14, entry_tid=15968], [object_number=83, tag_tid=14, entry_tid=15967], [object_number=82, tag_tid=14, entry_tid=15966], [object_number=81, tag_tid=14, entry_tid=15965]]], state=connected] 2024-04-23T21:25:59.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:59.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=80, tag_tid=14, entry_tid=15968' 2024-04-23T21:25:59.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=80, tag_tid=14, entry_tid=15968' 2024-04-23T21:25:59.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=80, tag_tid=14, entry_tid=15968' '!=' 'object_number=82, tag_tid=14, entry_tid=15766' 2024-04-23T21:25:59.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:25:59.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=80, tag_tid=14, entry_tid=15968' = 'object_number=82, tag_tid=14, entry_tid=15766' 2024-04-23T21:25:59.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=80, tag_tid=14, entry_tid=15968' 2024-04-23T21:25:59.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:25:59.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:25:59.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:25:59.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:25:59.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=80, tag_tid=14, entry_tid=15968' 2024-04-23T21:25:59.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:25:59.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:25:59.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:25:59.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:25:59.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:25:59.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:25:59.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:25:59.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=80, tag_tid=14, entry_tid=15968' 2024-04-23T21:25:59.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:25:59.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:25:59.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=15968 2024-04-23T21:25:59.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 15968 2024-04-23T21:25:59.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:25:59.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:25:59.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:25:59.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:25:59.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:59.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:25:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:25:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:25:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:25:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:25:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:25:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:25:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:25:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:25:59.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:25:59.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:25:59.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:59.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:25:59.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:25:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:25:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:25:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:25:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:25:59.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:25:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:25:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:25:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:25:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:25:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:25:59.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:25:59.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:59.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:59.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:59.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:25:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:25:59.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:25:59.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:25:59.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:59.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:25:59.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:25:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:25:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:25:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:25:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:25:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:59.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:25:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:25:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:25:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:25:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:25:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:25:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:25:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:25:59.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:25:59.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:59.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:59.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:25:59.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:25:59.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:25:59.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:25:59.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:59.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:59.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:59.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:25:59.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:59.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:59.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:59.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:25:59.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:25:59.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:25:59.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:25:59.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:59.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:25:59.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:59.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:25:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:25:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:25:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:25:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=80, tag_tid=14, entry_tid=16168], [object_number=83, tag_tid=14, entry_tid=16167], [object_number=82, tag_tid=14, entry_tid=16166], [object_number=81, tag_tid=14, entry_tid=16165]]], state=connected] 2024-04-23T21:25:59.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:59.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:25:59.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:25:59.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:59.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:59.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:59.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:25:59.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:25:59.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:25:59.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:25:59.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:25:59.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:25:59.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:25:59.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:25:59.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:59.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:25:59.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:59.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:25:59.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:25:59.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:25:59.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:25:59.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=80, tag_tid=14, entry_tid=16168], [object_number=83, tag_tid=14, entry_tid=16167], [object_number=82, tag_tid=14, entry_tid=16166], [object_number=81, tag_tid=14, entry_tid=16165]]], state=connected] 2024-04-23T21:25:59.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:25:59.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=80, tag_tid=14, entry_tid=16168' 2024-04-23T21:25:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=80, tag_tid=14, entry_tid=16168' 2024-04-23T21:25:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=80, tag_tid=14, entry_tid=16168' '!=' 'object_number=80, tag_tid=14, entry_tid=15968' 2024-04-23T21:25:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:25:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=80, tag_tid=14, entry_tid=16168' = 'object_number=80, tag_tid=14, entry_tid=15968' 2024-04-23T21:25:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=80, tag_tid=14, entry_tid=16168' 2024-04-23T21:25:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:25:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:25:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:25:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:25:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=80, tag_tid=14, entry_tid=16168' 2024-04-23T21:25:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:25:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:25:59.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:25:59.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:25:59.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:25:59.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:25:59.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:25:59.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=80, tag_tid=14, entry_tid=16168' 2024-04-23T21:25:59.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:25:59.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:25:59.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=16168 2024-04-23T21:25:59.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 16168 2024-04-23T21:25:59.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:25:59.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:25:59.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:25:59.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:25:59.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:59.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:25:59.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:25:59.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:25:59.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:25:59.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:25:59.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:25:59.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:25:59.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:59.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:25:59.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:25:59.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:25:59.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:25:59.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:25:59.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:25:59.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:25:59.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:25:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:25:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:25:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:25:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:25:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:25:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:25:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:25:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:25:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:25:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:25:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:25:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:59.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:25:59.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:59.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:25:59.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:25:59.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:25:59.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:25:59.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:25:59.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:25:59.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:25:59.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:25:59.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:25:59.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:25:59.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:25:59.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:25:59.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:25:59.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:25:59.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:25:59.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:25:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:25:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:00.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:00.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:00.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:00.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:00.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:00.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:00.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:00.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:00.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:00.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:00.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:00.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:00.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:00.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:00.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:00.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:00.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:00.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:00.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:00.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:00.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:00.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:00.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:00.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:00.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:00.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:00.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:00.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:00.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:00.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:00.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:00.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:00.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:00.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:00.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:00.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:00.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:00.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:00.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:00.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:00.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:00.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:00.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:00.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:00.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:00.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:00.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=80, tag_tid=14, entry_tid=16404], [object_number=83, tag_tid=14, entry_tid=16403], [object_number=82, tag_tid=14, entry_tid=16402], [object_number=81, tag_tid=14, entry_tid=16401]]], state=connected] 2024-04-23T21:26:00.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:00.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:00.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:00.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:00.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:00.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:00.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:00.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:00.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:00.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:00.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:00.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:00.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:00.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:00.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:00.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:00.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:00.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:00.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:00.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:00.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:00.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=80, tag_tid=14, entry_tid=16404], [object_number=83, tag_tid=14, entry_tid=16403], [object_number=82, tag_tid=14, entry_tid=16402], [object_number=81, tag_tid=14, entry_tid=16401]]], state=connected] 2024-04-23T21:26:00.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:00.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=80, tag_tid=14, entry_tid=16404' 2024-04-23T21:26:00.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=80, tag_tid=14, entry_tid=16404' 2024-04-23T21:26:00.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=80, tag_tid=14, entry_tid=16404' '!=' 'object_number=80, tag_tid=14, entry_tid=16168' 2024-04-23T21:26:00.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:00.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=80, tag_tid=14, entry_tid=16404' = 'object_number=80, tag_tid=14, entry_tid=16168' 2024-04-23T21:26:00.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=80, tag_tid=14, entry_tid=16404' 2024-04-23T21:26:00.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:00.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:00.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:00.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:00.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=80, tag_tid=14, entry_tid=16404' 2024-04-23T21:26:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:00.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:00.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=80, tag_tid=14, entry_tid=16404' 2024-04-23T21:26:00.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:00.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:00.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=16404 2024-04-23T21:26:00.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 16404 2024-04-23T21:26:00.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:00.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:00.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:00.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:00.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:00.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:00.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:00.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:00.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:00.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:00.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:00.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:00.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:00.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:00.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:00.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:00.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:00.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:00.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:00.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:00.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:00.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:00.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:00.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:00.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:00.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:00.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:00.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:00.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:00.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:00.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:00.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:00.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:00.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:00.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:00.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:00.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:00.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:00.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:00.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:00.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:00.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:00.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:00.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:00.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:00.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:00.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:00.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:00.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:00.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:00.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:00.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=81, tag_tid=14, entry_tid=16617], [object_number=80, tag_tid=14, entry_tid=16616], [object_number=83, tag_tid=14, entry_tid=16615], [object_number=82, tag_tid=14, entry_tid=16614]]], state=connected] 2024-04-23T21:26:00.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:00.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:00.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:00.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:00.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:00.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:00.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:00.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:00.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:00.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:00.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:00.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:00.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:00.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:00.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:00.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:00.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:00.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:00.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:00.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:00.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:00.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=81, tag_tid=14, entry_tid=16617], [object_number=80, tag_tid=14, entry_tid=16616], [object_number=83, tag_tid=14, entry_tid=16615], [object_number=82, tag_tid=14, entry_tid=16614]]], state=connected] 2024-04-23T21:26:00.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=81, tag_tid=14, entry_tid=16617' 2024-04-23T21:26:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=81, tag_tid=14, entry_tid=16617' 2024-04-23T21:26:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=81, tag_tid=14, entry_tid=16617' '!=' 'object_number=80, tag_tid=14, entry_tid=16404' 2024-04-23T21:26:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=81, tag_tid=14, entry_tid=16617' = 'object_number=80, tag_tid=14, entry_tid=16404' 2024-04-23T21:26:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=81, tag_tid=14, entry_tid=16617' 2024-04-23T21:26:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:00.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=81, tag_tid=14, entry_tid=16617' 2024-04-23T21:26:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:00.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:00.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:00.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:00.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:00.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:00.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:00.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=81, tag_tid=14, entry_tid=16617' 2024-04-23T21:26:00.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:00.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:00.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=16617 2024-04-23T21:26:00.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 16617 2024-04-23T21:26:00.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:00.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:00.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:00.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:00.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:00.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:00.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:00.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:00.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:00.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:00.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:00.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:00.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:00.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:00.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:00.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:00.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:00.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:00.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:00.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:00.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:00.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:00.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:00.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:00.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:00.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:00.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:00.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:00.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:00.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:00.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:00.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:00.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:00.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:00.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:00.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:00.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:00.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:00.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:00.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:00.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:00.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:00.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:00.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:00.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:00.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:00.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:00.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:00.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:00.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:00.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:00.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:00.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:00.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:00.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:00.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:01.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:01.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:01.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:01.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:01.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:01.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:01.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:01.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:01.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:01.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:01.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:01.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:01.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:01.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:01.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:01.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:01.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:01.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:01.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:01.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:01.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:01.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:01.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:01.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:01.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:01.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:01.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:01.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:01.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:01.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:01.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:01.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:01.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:01.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:01.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:01.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:01.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:01.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:01.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:01.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:01.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:01.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:01.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=80, tag_tid=14, entry_tid=16840], [object_number=83, tag_tid=14, entry_tid=16839], [object_number=82, tag_tid=14, entry_tid=16838], [object_number=81, tag_tid=14, entry_tid=16837]]], state=connected] 2024-04-23T21:26:01.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:01.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:01.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:01.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:01.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:01.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:01.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:01.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:01.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:01.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:01.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:01.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:01.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:01.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:01.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:01.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:01.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:01.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:01.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:01.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:01.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:01.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=80, tag_tid=14, entry_tid=16840], [object_number=83, tag_tid=14, entry_tid=16839], [object_number=82, tag_tid=14, entry_tid=16838], [object_number=81, tag_tid=14, entry_tid=16837]]], state=connected] 2024-04-23T21:26:01.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:01.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=80, tag_tid=14, entry_tid=16840' 2024-04-23T21:26:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=80, tag_tid=14, entry_tid=16840' 2024-04-23T21:26:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=80, tag_tid=14, entry_tid=16840' '!=' 'object_number=81, tag_tid=14, entry_tid=16617' 2024-04-23T21:26:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=80, tag_tid=14, entry_tid=16840' = 'object_number=81, tag_tid=14, entry_tid=16617' 2024-04-23T21:26:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=80, tag_tid=14, entry_tid=16840' 2024-04-23T21:26:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:01.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=80, tag_tid=14, entry_tid=16840' 2024-04-23T21:26:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:01.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:01.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:01.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:01.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:01.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:01.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:01.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=80, tag_tid=14, entry_tid=16840' 2024-04-23T21:26:01.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:01.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:01.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=16840 2024-04-23T21:26:01.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 16840 2024-04-23T21:26:01.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:01.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:01.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:01.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:01.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:01.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:01.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:01.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:01.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:01.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:01.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:01.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:01.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:01.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:01.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:01.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:01.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:01.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:01.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:01.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:01.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:01.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:01.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:01.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:01.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:01.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:01.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:01.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:01.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:01.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:01.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:01.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:01.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:01.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:01.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:01.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:01.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:01.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:01.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:01.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:01.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:01.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:01.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:01.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:01.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:01.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:01.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:01.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:01.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:01.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:01.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:01.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:01.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:01.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:01.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:01.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:01.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:01.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:01.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:01.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:01.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:01.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:01.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:01.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:01.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:01.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:01.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:01.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:01.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:01.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:01.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:01.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:01.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:01.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:01.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:01.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:01.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:01.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=83, tag_tid=14, entry_tid=17055], [object_number=82, tag_tid=14, entry_tid=17054], [object_number=81, tag_tid=14, entry_tid=17053], [object_number=80, tag_tid=14, entry_tid=17052]]], state=connected] 2024-04-23T21:26:01.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:01.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:01.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:01.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:01.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:01.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:01.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:01.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:01.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:01.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:01.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:01.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:01.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:01.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:01.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=83, tag_tid=14, entry_tid=17055], [object_number=82, tag_tid=14, entry_tid=17054], [object_number=81, tag_tid=14, entry_tid=17053], [object_number=80, tag_tid=14, entry_tid=17052]]], state=connected] 2024-04-23T21:26:01.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=83, tag_tid=14, entry_tid=17055' 2024-04-23T21:26:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=83, tag_tid=14, entry_tid=17055' 2024-04-23T21:26:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=83, tag_tid=14, entry_tid=17055' '!=' 'object_number=80, tag_tid=14, entry_tid=16840' 2024-04-23T21:26:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=83, tag_tid=14, entry_tid=17055' = 'object_number=80, tag_tid=14, entry_tid=16840' 2024-04-23T21:26:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=83, tag_tid=14, entry_tid=17055' 2024-04-23T21:26:01.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:01.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:01.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:01.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:01.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=83, tag_tid=14, entry_tid=17055' 2024-04-23T21:26:01.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:01.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:01.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:01.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:01.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:01.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:01.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:01.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=83, tag_tid=14, entry_tid=17055' 2024-04-23T21:26:01.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:01.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:01.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=17055 2024-04-23T21:26:01.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 17055 2024-04-23T21:26:01.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:01.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:01.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:01.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:01.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:01.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:01.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:01.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:01.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:01.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:01.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:01.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:01.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:01.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:01.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:01.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:01.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:01.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:01.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:01.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:01.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:01.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:01.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:01.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:01.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:01.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:01.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:01.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:01.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:01.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:01.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:01.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:01.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:01.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:01.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:01.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:01.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:01.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:01.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:01.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:01.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:01.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:01.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:02.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:02.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:02.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:02.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:02.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:02.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:02.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:02.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:02.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:02.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:02.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:02.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:02.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:02.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:02.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:02.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:02.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:02.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:02.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:02.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:02.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:02.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:02.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=83, tag_tid=14, entry_tid=17255], [object_number=82, tag_tid=14, entry_tid=17254], [object_number=81, tag_tid=14, entry_tid=17253], [object_number=80, tag_tid=14, entry_tid=17252]]], state=connected] 2024-04-23T21:26:02.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:02.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:02.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:02.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:02.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:02.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:02.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:02.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=83, tag_tid=14, entry_tid=17255], [object_number=82, tag_tid=14, entry_tid=17254], [object_number=81, tag_tid=14, entry_tid=17253], [object_number=80, tag_tid=14, entry_tid=17252]]], state=connected] 2024-04-23T21:26:02.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:02.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=83, tag_tid=14, entry_tid=17255' 2024-04-23T21:26:02.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=83, tag_tid=14, entry_tid=17255' 2024-04-23T21:26:02.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=83, tag_tid=14, entry_tid=17255' '!=' 'object_number=83, tag_tid=14, entry_tid=17055' 2024-04-23T21:26:02.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:02.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=83, tag_tid=14, entry_tid=17255' = 'object_number=83, tag_tid=14, entry_tid=17055' 2024-04-23T21:26:02.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=83, tag_tid=14, entry_tid=17255' 2024-04-23T21:26:02.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:02.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:02.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:02.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:02.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=83, tag_tid=14, entry_tid=17255' 2024-04-23T21:26:02.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:02.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:02.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:02.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=83, tag_tid=14, entry_tid=17255' 2024-04-23T21:26:02.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:02.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:02.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=17255 2024-04-23T21:26:02.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 17255 2024-04-23T21:26:02.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:02.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:02.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:02.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:02.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:02.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:02.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:02.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:02.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:02.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:02.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:02.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:02.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:02.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:02.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:02.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:02.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:02.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:02.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:02.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:02.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:02.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:02.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:02.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:02.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:02.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:02.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:02.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:02.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:02.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:02.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:02.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:02.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:02.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:02.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:02.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:02.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:02.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:02.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:02.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:02.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:02.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:02.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:02.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:02.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:02.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:02.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:02.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:02.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:02.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:02.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:02.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:02.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:02.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:02.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:02.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:02.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:02.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:02.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:02.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:02.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:02.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:02.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:02.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:02.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:02.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:02.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:02.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:02.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:02.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=83, tag_tid=14, entry_tid=17475], [object_number=82, tag_tid=14, entry_tid=17474], [object_number=81, tag_tid=14, entry_tid=17473], [object_number=80, tag_tid=14, entry_tid=17472]]], state=connected] 2024-04-23T21:26:02.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:02.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:02.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:02.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:02.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:02.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:02.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:02.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:02.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:02.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:02.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=83, tag_tid=14, entry_tid=17475], [object_number=82, tag_tid=14, entry_tid=17474], [object_number=81, tag_tid=14, entry_tid=17473], [object_number=80, tag_tid=14, entry_tid=17472]]], state=connected] 2024-04-23T21:26:02.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:02.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=83, tag_tid=14, entry_tid=17475' 2024-04-23T21:26:02.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=83, tag_tid=14, entry_tid=17475' 2024-04-23T21:26:02.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=83, tag_tid=14, entry_tid=17475' '!=' 'object_number=83, tag_tid=14, entry_tid=17255' 2024-04-23T21:26:02.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:02.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=83, tag_tid=14, entry_tid=17475' = 'object_number=83, tag_tid=14, entry_tid=17255' 2024-04-23T21:26:02.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=83, tag_tid=14, entry_tid=17475' 2024-04-23T21:26:02.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:02.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:02.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:02.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:02.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=83, tag_tid=14, entry_tid=17475' 2024-04-23T21:26:02.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:02.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:02.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:02.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:02.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:02.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:02.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:02.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=83, tag_tid=14, entry_tid=17475' 2024-04-23T21:26:02.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:02.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:02.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=17475 2024-04-23T21:26:02.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 17475 2024-04-23T21:26:02.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:02.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:02.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:02.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:02.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:02.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:02.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:02.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:02.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:02.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:02.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:02.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:02.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:02.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:02.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:02.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:02.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:02.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:02.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:02.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:02.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:02.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:02.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:02.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:02.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:02.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:02.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:02.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:02.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:02.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:02.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:02.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:02.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:02.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:02.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:02.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:02.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:02.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:02.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:02.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:02.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:02.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:02.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:02.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:02.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:02.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:02.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:02.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:03.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:03.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:03.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:03.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:03.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:03.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:03.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:03.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:03.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:03.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:03.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:03.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:03.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:03.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:03.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:03.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:03.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:03.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:03.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:03.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:03.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:03.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:03.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:03.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:03.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:03.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:03.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:03.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:03.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:03.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:03.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:03.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:03.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:03.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:03.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:03.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:03.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:03.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:03.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:03.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:03.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:03.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:03.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:03.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:03.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:03.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:03.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:03.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:03.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:03.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:03.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:03.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:03.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:03.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:03.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:03.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:03.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:03.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:03.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:03.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=81, tag_tid=14, entry_tid=17685], [object_number=80, tag_tid=14, entry_tid=17684], [object_number=83, tag_tid=14, entry_tid=17683], [object_number=82, tag_tid=14, entry_tid=17682]]], state=connected] 2024-04-23T21:26:03.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:03.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:03.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:03.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:03.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:03.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:03.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:03.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:03.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:03.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:03.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:03.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:03.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:03.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:03.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:03.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:03.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:03.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:03.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:03.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:03.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:03.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=81, tag_tid=14, entry_tid=17685], [object_number=80, tag_tid=14, entry_tid=17684], [object_number=83, tag_tid=14, entry_tid=17683], [object_number=82, tag_tid=14, entry_tid=17682]]], state=connected] 2024-04-23T21:26:03.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:03.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=81, tag_tid=14, entry_tid=17685' 2024-04-23T21:26:03.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=81, tag_tid=14, entry_tid=17685' 2024-04-23T21:26:03.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=81, tag_tid=14, entry_tid=17685' '!=' 'object_number=83, tag_tid=14, entry_tid=17475' 2024-04-23T21:26:03.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:03.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=81, tag_tid=14, entry_tid=17685' = 'object_number=83, tag_tid=14, entry_tid=17475' 2024-04-23T21:26:03.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=81, tag_tid=14, entry_tid=17685' 2024-04-23T21:26:03.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:03.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:03.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:03.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:03.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=81, tag_tid=14, entry_tid=17685' 2024-04-23T21:26:03.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:03.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:03.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:03.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:03.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:03.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:03.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:03.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=81, tag_tid=14, entry_tid=17685' 2024-04-23T21:26:03.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:03.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:03.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=17685 2024-04-23T21:26:03.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 17685 2024-04-23T21:26:03.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:03.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:03.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:03.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:03.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:03.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:03.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:03.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:03.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:03.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:03.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:03.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:03.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:03.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:03.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:03.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:03.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:03.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:03.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:03.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:03.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:03.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:03.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:03.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:03.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:03.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:03.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:03.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:03.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:03.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:03.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:03.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:03.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:03.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:03.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:03.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:03.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:03.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:03.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:03.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:03.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:03.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:03.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:03.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:03.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:03.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:03.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:03.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:03.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:03.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:03.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:03.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:03.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:03.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:03.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:03.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:03.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:03.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:03.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:03.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:03.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:03.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:03.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:03.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:03.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:03.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:03.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:03.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:03.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:03.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:03.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:03.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:03.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:03.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:03.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:03.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:03.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:03.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:03.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:03.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:03.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:03.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:03.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:03.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:03.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:03.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:03.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:03.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:03.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:03.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:03.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:03.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:03.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:03.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:03.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:03.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:03.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:03.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:03.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:03.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:03.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:03.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=80, tag_tid=14, entry_tid=17884], [object_number=83, tag_tid=14, entry_tid=17883], [object_number=82, tag_tid=14, entry_tid=17882], [object_number=81, tag_tid=14, entry_tid=17881]]], state=connected] 2024-04-23T21:26:03.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:03.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:03.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:03.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:03.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:03.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:03.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:03.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:03.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:03.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:03.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:03.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:03.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:03.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:03.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:03.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:03.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:03.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=80, tag_tid=14, entry_tid=17884], [object_number=83, tag_tid=14, entry_tid=17883], [object_number=82, tag_tid=14, entry_tid=17882], [object_number=81, tag_tid=14, entry_tid=17881]]], state=connected] 2024-04-23T21:26:03.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:03.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=80, tag_tid=14, entry_tid=17884' 2024-04-23T21:26:03.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=80, tag_tid=14, entry_tid=17884' 2024-04-23T21:26:03.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=80, tag_tid=14, entry_tid=17884' '!=' 'object_number=81, tag_tid=14, entry_tid=17685' 2024-04-23T21:26:03.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:03.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=80, tag_tid=14, entry_tid=17884' = 'object_number=81, tag_tid=14, entry_tid=17685' 2024-04-23T21:26:03.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=80, tag_tid=14, entry_tid=17884' 2024-04-23T21:26:03.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:03.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:03.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:03.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:03.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=80, tag_tid=14, entry_tid=17884' 2024-04-23T21:26:03.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:03.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:03.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:03.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:03.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:03.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=80, tag_tid=14, entry_tid=17884' 2024-04-23T21:26:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:03.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:03.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=17884 2024-04-23T21:26:03.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 17884 2024-04-23T21:26:03.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:03.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:03.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:04.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:04.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:04.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:04.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:04.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:04.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:04.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:04.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:04.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:04.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:04.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:04.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:04.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:04.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:04.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:04.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:04.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:04.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:04.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:04.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:04.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:04.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:04.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:04.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:04.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:04.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:04.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:04.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:04.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:04.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:04.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:04.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:04.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:04.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:04.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:04.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:04.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:04.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:04.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:04.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:04.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:04.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:04.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:04.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:04.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:04.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:04.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:04.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:04.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:04.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:04.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:04.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:04.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:04.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:04.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:04.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:04.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:04.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:04.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:04.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:04.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:04.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:04.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:04.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:04.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:04.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:04.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:04.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:04.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:04.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:04.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:04.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:04.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:04.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:04.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:04.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:04.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:04.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:04.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:04.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:04.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:04.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:04.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:04.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:04.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:04.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:04.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:04.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:04.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:04.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:04.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:04.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:04.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:04.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:04.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:04.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:04.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:04.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:04.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:04.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:04.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:04.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:04.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:04.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:04.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:04.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=82, tag_tid=14, entry_tid=18138], [object_number=81, tag_tid=14, entry_tid=18137], [object_number=80, tag_tid=14, entry_tid=18136], [object_number=83, tag_tid=14, entry_tid=18135]]], state=connected] 2024-04-23T21:26:04.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:04.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:04.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:04.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:04.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:04.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:04.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:04.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:04.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:04.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:04.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:04.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:04.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:04.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:04.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:04.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:04.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:04.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:04.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:04.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:04.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:04.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=82, tag_tid=14, entry_tid=18138], [object_number=81, tag_tid=14, entry_tid=18137], [object_number=80, tag_tid=14, entry_tid=18136], [object_number=83, tag_tid=14, entry_tid=18135]]], state=connected] 2024-04-23T21:26:04.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:04.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=82, tag_tid=14, entry_tid=18138' 2024-04-23T21:26:04.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=82, tag_tid=14, entry_tid=18138' 2024-04-23T21:26:04.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=82, tag_tid=14, entry_tid=18138' '!=' 'object_number=80, tag_tid=14, entry_tid=17884' 2024-04-23T21:26:04.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:04.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=82, tag_tid=14, entry_tid=18138' = 'object_number=80, tag_tid=14, entry_tid=17884' 2024-04-23T21:26:04.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=82, tag_tid=14, entry_tid=18138' 2024-04-23T21:26:04.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:04.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:04.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:04.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:04.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=82, tag_tid=14, entry_tid=18138' 2024-04-23T21:26:04.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:04.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:04.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:04.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:04.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:04.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:04.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:04.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=82, tag_tid=14, entry_tid=18138' 2024-04-23T21:26:04.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:04.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:04.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=18138 2024-04-23T21:26:04.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 18138 2024-04-23T21:26:04.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:04.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:04.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:04.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:04.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:04.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:04.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:04.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:04.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:04.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:04.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:04.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:04.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:04.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:04.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:04.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:04.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:04.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:04.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:04.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:04.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:04.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:04.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:04.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:04.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:04.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:04.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:04.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:04.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:04.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:04.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:04.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:04.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:04.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:04.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:04.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:04.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:04.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:04.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:04.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:04.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:04.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:04.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:04.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:04.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:04.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:04.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:04.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:04.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:04.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:04.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:04.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:04.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:04.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:04.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:04.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=80, tag_tid=14, entry_tid=18340], [object_number=83, tag_tid=14, entry_tid=18339], [object_number=82, tag_tid=14, entry_tid=18338], [object_number=81, tag_tid=14, entry_tid=18337]]], state=connected] 2024-04-23T21:26:04.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:04.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:04.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:04.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:04.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:04.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:04.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:04.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:04.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:04.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:04.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:04.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:04.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:04.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:04.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:04.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:04.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:04.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:04.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:04.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:04.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:04.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=80, tag_tid=14, entry_tid=18340], [object_number=83, tag_tid=14, entry_tid=18339], [object_number=82, tag_tid=14, entry_tid=18338], [object_number=81, tag_tid=14, entry_tid=18337]]], state=connected] 2024-04-23T21:26:04.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:04.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=80, tag_tid=14, entry_tid=18340' 2024-04-23T21:26:04.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=80, tag_tid=14, entry_tid=18340' 2024-04-23T21:26:04.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=80, tag_tid=14, entry_tid=18340' '!=' 'object_number=82, tag_tid=14, entry_tid=18138' 2024-04-23T21:26:04.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:04.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=80, tag_tid=14, entry_tid=18340' = 'object_number=82, tag_tid=14, entry_tid=18138' 2024-04-23T21:26:04.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=80, tag_tid=14, entry_tid=18340' 2024-04-23T21:26:04.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:04.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:04.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:04.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:04.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=80, tag_tid=14, entry_tid=18340' 2024-04-23T21:26:04.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:04.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=80, tag_tid=14, entry_tid=18340' 2024-04-23T21:26:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=18340 2024-04-23T21:26:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 18340 2024-04-23T21:26:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:04.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:04.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:05.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:05.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:05.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:05.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:05.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:05.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:05.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:05.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:05.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:05.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:05.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:05.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:05.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:05.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:05.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:05.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:05.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:05.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:05.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:05.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:05.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:05.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:05.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:05.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:05.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:05.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:05.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:05.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:05.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:05.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:05.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:05.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:05.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:05.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:05.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:05.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:05.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:05.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:05.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:05.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:05.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:05.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:05.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:05.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:05.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:05.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:05.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:05.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:05.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:05.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:05.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:05.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:05.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:05.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:05.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:05.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:05.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:05.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:05.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:05.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:05.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:05.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:05.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:05.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:05.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:05.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:05.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:05.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:05.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=81, tag_tid=14, entry_tid=18553], [object_number=80, tag_tid=14, entry_tid=18552], [object_number=83, tag_tid=14, entry_tid=18551], [object_number=82, tag_tid=14, entry_tid=18550]]], state=connected] 2024-04-23T21:26:05.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:05.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:05.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:05.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:05.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:05.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:05.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:05.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:05.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:05.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:05.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:05.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:05.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:05.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:05.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:05.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:05.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:05.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:05.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:05.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=81, tag_tid=14, entry_tid=18553], [object_number=80, tag_tid=14, entry_tid=18552], [object_number=83, tag_tid=14, entry_tid=18551], [object_number=82, tag_tid=14, entry_tid=18550]]], state=connected] 2024-04-23T21:26:05.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:05.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=81, tag_tid=14, entry_tid=18553' 2024-04-23T21:26:05.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=81, tag_tid=14, entry_tid=18553' 2024-04-23T21:26:05.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=81, tag_tid=14, entry_tid=18553' '!=' 'object_number=80, tag_tid=14, entry_tid=18340' 2024-04-23T21:26:05.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:05.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=81, tag_tid=14, entry_tid=18553' = 'object_number=80, tag_tid=14, entry_tid=18340' 2024-04-23T21:26:05.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=81, tag_tid=14, entry_tid=18553' 2024-04-23T21:26:05.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:05.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:05.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:05.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:05.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=81, tag_tid=14, entry_tid=18553' 2024-04-23T21:26:05.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:05.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:05.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:05.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:05.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:05.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:05.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:05.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=81, tag_tid=14, entry_tid=18553' 2024-04-23T21:26:05.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:05.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:05.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=18553 2024-04-23T21:26:05.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 18553 2024-04-23T21:26:05.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:05.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:05.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:05.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:05.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:05.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:05.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:05.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:05.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:05.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:05.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:05.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:05.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:05.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:05.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:05.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:05.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:05.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:05.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:05.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:05.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:05.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:05.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:05.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:05.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:05.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:05.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:05.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:05.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:05.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:05.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:05.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:05.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:05.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:05.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:05.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:05.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:05.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:05.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:05.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:05.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:05.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:05.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:05.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:05.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:05.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:05.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:05.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:05.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=81, tag_tid=14, entry_tid=18749], [object_number=80, tag_tid=14, entry_tid=18748], [object_number=83, tag_tid=14, entry_tid=18747], [object_number=82, tag_tid=14, entry_tid=18746]]], state=connected] 2024-04-23T21:26:05.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:05.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:05.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:05.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:05.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:05.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:05.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:05.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:05.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:05.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:05.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:05.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:05.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:05.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:05.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:05.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:05.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:05.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:05.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:05.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:05.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:05.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=81, tag_tid=14, entry_tid=18749], [object_number=80, tag_tid=14, entry_tid=18748], [object_number=83, tag_tid=14, entry_tid=18747], [object_number=82, tag_tid=14, entry_tid=18746]]], state=connected] 2024-04-23T21:26:05.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=81, tag_tid=14, entry_tid=18749' 2024-04-23T21:26:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=81, tag_tid=14, entry_tid=18749' 2024-04-23T21:26:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=81, tag_tid=14, entry_tid=18749' '!=' 'object_number=81, tag_tid=14, entry_tid=18553' 2024-04-23T21:26:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=81, tag_tid=14, entry_tid=18749' = 'object_number=81, tag_tid=14, entry_tid=18553' 2024-04-23T21:26:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=81, tag_tid=14, entry_tid=18749' 2024-04-23T21:26:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:05.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:05.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=81, tag_tid=14, entry_tid=18749' 2024-04-23T21:26:05.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:05.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:05.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:05.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:05.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:05.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:05.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:05.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=81, tag_tid=14, entry_tid=18749' 2024-04-23T21:26:05.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:05.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:05.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=18749 2024-04-23T21:26:05.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 18749 2024-04-23T21:26:05.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:05.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:05.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:06.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:06.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:06.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:06.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:06.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:06.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:06.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:06.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:06.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:06.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:06.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:06.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:06.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:06.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:06.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:06.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:06.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:06.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:06.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:06.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:06.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:06.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:06.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:06.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:06.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:06.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:06.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:06.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:06.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:06.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:06.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:06.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:06.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:06.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:06.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:06.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:06.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:06.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:06.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:06.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:06.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:06.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:06.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:06.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:06.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:06.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:06.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:06.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:06.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:06.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:06.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:06.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:06.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:06.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:06.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:06.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:06.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:06.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:06.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:06.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:06.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:06.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:06.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:06.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:06.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:06.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:06.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:06.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:06.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:06.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:06.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:06.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:06.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:06.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:06.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=82, tag_tid=14, entry_tid=18950], [object_number=81, tag_tid=14, entry_tid=18949], [object_number=80, tag_tid=14, entry_tid=18948], [object_number=83, tag_tid=14, entry_tid=18947]]], state=connected] 2024-04-23T21:26:06.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:06.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:06.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:06.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:06.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:06.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:06.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:06.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:06.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 20 2024-04-23T21:26:06.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:06.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:06.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:06.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=82, tag_tid=14, entry_tid=18950], [object_number=81, tag_tid=14, entry_tid=18949], [object_number=80, tag_tid=14, entry_tid=18948], [object_number=83, tag_tid=14, entry_tid=18947]]], state=connected] 2024-04-23T21:26:06.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:06.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=82, tag_tid=14, entry_tid=18950' 2024-04-23T21:26:06.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=82, tag_tid=14, entry_tid=18950' 2024-04-23T21:26:06.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=82, tag_tid=14, entry_tid=18950' '!=' 'object_number=81, tag_tid=14, entry_tid=18749' 2024-04-23T21:26:06.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:06.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=82, tag_tid=14, entry_tid=18950' = 'object_number=81, tag_tid=14, entry_tid=18749' 2024-04-23T21:26:06.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=82, tag_tid=14, entry_tid=18950' 2024-04-23T21:26:06.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:06.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:06.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:06.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=82, tag_tid=14, entry_tid=18950' 2024-04-23T21:26:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:06.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:06.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:06.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:06.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:06.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=82, tag_tid=14, entry_tid=18950' 2024-04-23T21:26:06.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:06.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:06.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=18950 2024-04-23T21:26:06.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 18950 2024-04-23T21:26:06.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:06.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:06.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:06.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:06.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:06.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:06.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:06.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:06.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:06.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:06.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:06.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:06.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:06.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:06.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:06.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:06.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:06.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:06.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:06.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:06.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:06.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:06.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:06.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:06.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:06.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:06.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:06.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:06.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:06.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:06.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:06.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:06.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:06.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:06.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:06.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:06.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:06.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:06.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:06.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:06.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:06.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:06.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:06.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:06.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:06.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:06.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:06.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:06.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:06.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:06.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:06.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:06.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:06.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:06.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:06.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:06.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:06.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:06.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:06.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:06.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:06.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:06.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:06.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:06.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:06.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:06.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:06.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:06.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:06.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:06.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:06.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:06.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:06.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:06.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:06.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:06.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:06.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:06.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:06.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:06.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=19114], [object_number=85, tag_tid=14, entry_tid=19113], [object_number=84, tag_tid=14, entry_tid=19112], [object_number=87, tag_tid=14, entry_tid=19111]]], state=connected] 2024-04-23T21:26:06.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:06.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:06.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:06.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:06.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:06.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:06.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:06.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:06.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:06.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:06.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:06.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:06.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:06.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:06.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:06.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:06.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:06.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:06.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:06.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:06.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:06.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=19114], [object_number=85, tag_tid=14, entry_tid=19113], [object_number=84, tag_tid=14, entry_tid=19112], [object_number=87, tag_tid=14, entry_tid=19111]]], state=connected] 2024-04-23T21:26:06.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:06.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=86, tag_tid=14, entry_tid=19114' 2024-04-23T21:26:06.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=86, tag_tid=14, entry_tid=19114' 2024-04-23T21:26:06.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=86, tag_tid=14, entry_tid=19114' '!=' 'object_number=82, tag_tid=14, entry_tid=18950' 2024-04-23T21:26:06.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:06.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=86, tag_tid=14, entry_tid=19114' = 'object_number=82, tag_tid=14, entry_tid=18950' 2024-04-23T21:26:06.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=86, tag_tid=14, entry_tid=19114' 2024-04-23T21:26:06.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:06.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:06.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:06.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:06.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=86, tag_tid=14, entry_tid=19114' 2024-04-23T21:26:06.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:06.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:06.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:06.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:06.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:06.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:06.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:06.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=86, tag_tid=14, entry_tid=19114' 2024-04-23T21:26:06.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:06.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:06.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=19114 2024-04-23T21:26:06.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 19114 2024-04-23T21:26:06.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:06.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:06.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:07.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:07.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:07.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:07.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:07.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:07.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:07.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:07.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:07.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:07.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:07.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:07.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:07.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:07.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:07.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:07.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:07.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:07.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:07.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:07.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:07.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:07.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:07.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:07.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:07.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:07.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:07.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:07.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:07.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:07.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:07.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:07.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:07.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:07.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:07.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:07.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:07.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:07.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:07.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:07.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:07.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:07.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:07.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:07.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:07.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:07.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:07.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:07.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:07.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:07.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:07.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:07.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:07.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:07.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:07.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:07.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:07.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:07.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:07.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:07.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:07.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:07.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:07.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:07.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:07.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:07.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:07.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:07.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:07.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:07.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:07.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:07.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:07.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:07.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:07.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:07.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:07.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:07.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:07.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:07.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:07.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:07.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:07.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=87, tag_tid=14, entry_tid=19299], [object_number=86, tag_tid=14, entry_tid=19298], [object_number=85, tag_tid=14, entry_tid=19297], [object_number=84, tag_tid=14, entry_tid=19296]]], state=connected] 2024-04-23T21:26:07.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:07.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:07.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:07.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:07.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:07.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:07.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:07.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:07.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:07.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:07.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:07.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:07.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:07.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:07.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:07.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:07.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:07.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:07.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:07.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:07.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:07.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=87, tag_tid=14, entry_tid=19299], [object_number=86, tag_tid=14, entry_tid=19298], [object_number=85, tag_tid=14, entry_tid=19297], [object_number=84, tag_tid=14, entry_tid=19296]]], state=connected] 2024-04-23T21:26:07.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:07.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=87, tag_tid=14, entry_tid=19299' 2024-04-23T21:26:07.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=87, tag_tid=14, entry_tid=19299' 2024-04-23T21:26:07.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=87, tag_tid=14, entry_tid=19299' '!=' 'object_number=86, tag_tid=14, entry_tid=19114' 2024-04-23T21:26:07.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:07.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=87, tag_tid=14, entry_tid=19299' = 'object_number=86, tag_tid=14, entry_tid=19114' 2024-04-23T21:26:07.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=87, tag_tid=14, entry_tid=19299' 2024-04-23T21:26:07.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:07.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:07.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:07.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:07.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=87, tag_tid=14, entry_tid=19299' 2024-04-23T21:26:07.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:07.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:07.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:07.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:07.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=87, tag_tid=14, entry_tid=19299' 2024-04-23T21:26:07.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:07.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=19299 2024-04-23T21:26:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 19299 2024-04-23T21:26:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:07.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:07.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:07.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:07.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:07.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:07.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:07.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:07.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:07.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:07.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:07.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:07.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:07.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:07.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:07.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:07.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:07.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:07.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:07.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:07.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:07.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:07.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:07.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:07.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:07.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:07.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:07.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:07.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:07.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:07.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:07.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:07.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:07.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:07.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:07.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:07.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:07.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:07.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:07.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:07.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:07.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:07.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:07.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:07.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:07.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:07.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:07.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:07.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:07.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:07.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:07.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:07.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:07.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:07.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:07.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:07.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:07.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:07.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:07.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:07.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:07.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:07.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:07.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:07.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:07.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:07.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:07.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:07.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:07.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:07.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:07.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:07.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:07.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:07.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:07.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:07.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:07.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:07.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:07.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:07.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:07.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:07.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:07.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:07.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:07.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:07.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:07.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:07.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:07.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:07.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:07.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:07.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:07.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:07.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=87, tag_tid=14, entry_tid=19479], [object_number=86, tag_tid=14, entry_tid=19478], [object_number=85, tag_tid=14, entry_tid=19477], [object_number=84, tag_tid=14, entry_tid=19476]]], state=connected] 2024-04-23T21:26:07.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:07.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:07.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:07.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:07.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:07.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:07.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:07.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:07.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:07.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:07.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:07.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:07.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:07.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:07.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:07.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:07.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:07.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:07.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:07.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:07.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:07.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=87, tag_tid=14, entry_tid=19479], [object_number=86, tag_tid=14, entry_tid=19478], [object_number=85, tag_tid=14, entry_tid=19477], [object_number=84, tag_tid=14, entry_tid=19476]]], state=connected] 2024-04-23T21:26:07.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:07.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=87, tag_tid=14, entry_tid=19479' 2024-04-23T21:26:07.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=87, tag_tid=14, entry_tid=19479' 2024-04-23T21:26:07.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=87, tag_tid=14, entry_tid=19479' '!=' 'object_number=87, tag_tid=14, entry_tid=19299' 2024-04-23T21:26:07.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:07.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=87, tag_tid=14, entry_tid=19479' = 'object_number=87, tag_tid=14, entry_tid=19299' 2024-04-23T21:26:07.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=87, tag_tid=14, entry_tid=19479' 2024-04-23T21:26:07.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:07.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:07.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:07.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:07.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=87, tag_tid=14, entry_tid=19479' 2024-04-23T21:26:07.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:07.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:07.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:07.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:07.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:07.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:07.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:07.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=87, tag_tid=14, entry_tid=19479' 2024-04-23T21:26:07.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:07.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:07.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=19479 2024-04-23T21:26:07.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 19479 2024-04-23T21:26:07.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:07.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:07.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:08.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:08.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:08.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:08.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:08.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:08.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:08.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:08.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:08.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:08.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:08.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:08.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:08.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:08.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:08.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:08.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:08.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:08.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:08.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:08.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:08.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:08.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:08.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:08.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:08.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:08.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:08.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:08.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:08.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:08.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:08.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:08.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:08.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:08.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:08.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:08.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:08.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:08.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:08.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:08.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:08.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:08.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:08.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:08.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:08.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:08.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:08.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:08.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:08.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:08.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:08.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:08.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:08.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:08.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:08.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:08.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:08.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:08.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:08.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:08.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:08.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:08.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:08.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:08.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:08.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:08.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:08.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:08.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:08.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:08.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:08.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:08.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=19670], [object_number=85, tag_tid=14, entry_tid=19669], [object_number=84, tag_tid=14, entry_tid=19668], [object_number=87, tag_tid=14, entry_tid=19667]]], state=connected] 2024-04-23T21:26:08.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:08.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:08.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:08.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=19670], [object_number=85, tag_tid=14, entry_tid=19669], [object_number=84, tag_tid=14, entry_tid=19668], [object_number=87, tag_tid=14, entry_tid=19667]]], state=connected] 2024-04-23T21:26:08.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:08.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=86, tag_tid=14, entry_tid=19670' 2024-04-23T21:26:08.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=86, tag_tid=14, entry_tid=19670' 2024-04-23T21:26:08.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=86, tag_tid=14, entry_tid=19670' '!=' 'object_number=87, tag_tid=14, entry_tid=19479' 2024-04-23T21:26:08.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:08.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=86, tag_tid=14, entry_tid=19670' = 'object_number=87, tag_tid=14, entry_tid=19479' 2024-04-23T21:26:08.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=86, tag_tid=14, entry_tid=19670' 2024-04-23T21:26:08.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:08.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:08.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:08.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:08.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=86, tag_tid=14, entry_tid=19670' 2024-04-23T21:26:08.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:08.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:08.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:08.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:08.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:08.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:08.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:08.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=86, tag_tid=14, entry_tid=19670' 2024-04-23T21:26:08.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:08.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:08.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=19670 2024-04-23T21:26:08.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 19670 2024-04-23T21:26:08.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:08.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:08.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:08.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:08.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:08.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:08.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:08.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:08.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:08.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:08.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:08.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:08.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:08.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:08.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:08.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:08.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:08.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:08.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:08.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:08.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:08.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:08.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:08.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:08.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:08.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:08.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:08.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:08.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:08.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:08.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:08.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:08.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:08.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:08.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:08.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:08.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:08.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:08.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:08.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:08.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:08.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:08.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:08.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:08.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:08.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:08.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:08.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:08.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:08.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:08.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:08.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:08.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:08.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:08.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:08.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:08.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:08.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:08.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:08.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:08.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:08.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:08.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:08.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:08.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:08.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:08.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:08.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:08.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:08.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:08.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:08.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:08.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:08.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:08.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=84, tag_tid=14, entry_tid=19888], [object_number=87, tag_tid=14, entry_tid=19887], [object_number=86, tag_tid=14, entry_tid=19886], [object_number=85, tag_tid=14, entry_tid=19885]]], state=connected] 2024-04-23T21:26:08.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:08.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:08.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:08.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:08.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:08.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:08.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:08.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:08.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:08.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:08.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:08.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:08.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:08.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:08.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:08.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:08.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:08.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:08.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:08.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:08.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:08.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=84, tag_tid=14, entry_tid=19888], [object_number=87, tag_tid=14, entry_tid=19887], [object_number=86, tag_tid=14, entry_tid=19886], [object_number=85, tag_tid=14, entry_tid=19885]]], state=connected] 2024-04-23T21:26:09.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:09.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=84, tag_tid=14, entry_tid=19888' 2024-04-23T21:26:09.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=84, tag_tid=14, entry_tid=19888' 2024-04-23T21:26:09.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=84, tag_tid=14, entry_tid=19888' '!=' 'object_number=86, tag_tid=14, entry_tid=19670' 2024-04-23T21:26:09.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:09.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=84, tag_tid=14, entry_tid=19888' = 'object_number=86, tag_tid=14, entry_tid=19670' 2024-04-23T21:26:09.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=84, tag_tid=14, entry_tid=19888' 2024-04-23T21:26:09.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:09.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:09.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:09.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:09.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=84, tag_tid=14, entry_tid=19888' 2024-04-23T21:26:09.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:09.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:09.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:09.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:09.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:09.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:09.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:09.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=84, tag_tid=14, entry_tid=19888' 2024-04-23T21:26:09.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:09.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:09.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=19888 2024-04-23T21:26:09.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 19888 2024-04-23T21:26:09.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:09.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:09.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:09.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:09.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:09.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:09.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:09.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:09.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:09.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:09.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:09.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:09.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:09.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:09.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:09.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:09.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:09.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:09.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:09.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:09.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:09.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:09.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:09.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:09.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:09.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:09.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:09.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:09.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:09.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:09.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:09.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:09.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:09.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:09.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:09.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:09.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:09.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:09.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:09.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:09.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:09.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:09.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:09.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:09.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:09.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:09.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:09.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:09.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:09.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:09.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:09.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:09.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:09.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:09.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:09.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:09.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:09.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:09.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:09.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:09.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:09.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:09.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:09.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:09.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:09.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:09.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:09.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:09.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:09.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:09.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:09.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:09.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:09.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:09.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:09.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:09.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:09.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:09.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:09.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:09.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:09.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:09.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:09.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:09.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:09.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:09.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:09.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:09.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=20110], [object_number=85, tag_tid=14, entry_tid=20109], [object_number=84, tag_tid=14, entry_tid=20108], [object_number=87, tag_tid=14, entry_tid=20107]]], state=connected] 2024-04-23T21:26:09.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:09.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:09.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:09.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:09.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:09.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:09.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:09.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:09.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=20110], [object_number=85, tag_tid=14, entry_tid=20109], [object_number=84, tag_tid=14, entry_tid=20108], [object_number=87, tag_tid=14, entry_tid=20107]]], state=connected] 2024-04-23T21:26:09.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:09.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=86, tag_tid=14, entry_tid=20110' 2024-04-23T21:26:09.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=86, tag_tid=14, entry_tid=20110' 2024-04-23T21:26:09.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=86, tag_tid=14, entry_tid=20110' '!=' 'object_number=84, tag_tid=14, entry_tid=19888' 2024-04-23T21:26:09.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:09.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=86, tag_tid=14, entry_tid=20110' = 'object_number=84, tag_tid=14, entry_tid=19888' 2024-04-23T21:26:09.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=86, tag_tid=14, entry_tid=20110' 2024-04-23T21:26:09.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:09.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:09.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:09.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:09.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=86, tag_tid=14, entry_tid=20110' 2024-04-23T21:26:09.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:09.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:09.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:09.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:09.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:09.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:09.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:09.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=86, tag_tid=14, entry_tid=20110' 2024-04-23T21:26:09.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:09.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:09.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=20110 2024-04-23T21:26:09.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 20110 2024-04-23T21:26:09.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:09.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:09.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:09.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:09.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:09.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:09.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:09.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:09.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:09.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:09.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:09.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:09.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:09.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:09.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:09.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:09.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:09.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:09.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:09.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:09.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:09.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:09.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:09.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:09.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:09.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:09.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:09.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:09.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:09.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:09.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:09.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:09.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:09.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:09.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:09.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:09.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:09.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:09.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:09.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:09.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:09.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:09.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:09.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:09.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:09.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:09.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:09.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:09.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:09.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:09.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:09.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:09.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:09.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:09.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:09.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:09.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:09.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:09.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:09.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:09.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:09.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:09.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:09.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:09.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:09.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:09.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:09.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:09.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:09.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:09.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:09.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:09.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:09.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:09.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:09.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:09.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:09.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:09.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:09.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:09.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:09.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:09.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:09.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:09.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:09.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:09.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:09.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:09.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:09.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:09.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:09.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:09.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:09.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:09.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:09.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:09.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:09.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:09.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:09.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:09.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:09.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:09.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:09.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:09.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=20350], [object_number=85, tag_tid=14, entry_tid=20349], [object_number=84, tag_tid=14, entry_tid=20348], [object_number=87, tag_tid=14, entry_tid=20347]]], state=connected] 2024-04-23T21:26:09.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:09.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:09.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:09.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:09.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:09.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:09.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:09.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:09.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:09.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:09.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:09.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:09.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:09.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:09.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:09.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:09.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:10.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:10.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:10.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:10.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:10.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=20350], [object_number=85, tag_tid=14, entry_tid=20349], [object_number=84, tag_tid=14, entry_tid=20348], [object_number=87, tag_tid=14, entry_tid=20347]]], state=connected] 2024-04-23T21:26:10.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:10.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=86, tag_tid=14, entry_tid=20350' 2024-04-23T21:26:10.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=86, tag_tid=14, entry_tid=20350' 2024-04-23T21:26:10.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=86, tag_tid=14, entry_tid=20350' '!=' 'object_number=86, tag_tid=14, entry_tid=20110' 2024-04-23T21:26:10.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:10.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=86, tag_tid=14, entry_tid=20350' = 'object_number=86, tag_tid=14, entry_tid=20110' 2024-04-23T21:26:10.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=86, tag_tid=14, entry_tid=20350' 2024-04-23T21:26:10.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:10.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:10.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:10.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:10.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=86, tag_tid=14, entry_tid=20350' 2024-04-23T21:26:10.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:10.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:10.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:10.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:10.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:10.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:10.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:10.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=86, tag_tid=14, entry_tid=20350' 2024-04-23T21:26:10.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:10.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:10.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=20350 2024-04-23T21:26:10.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 20350 2024-04-23T21:26:10.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:10.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:10.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:10.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:10.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:10.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:10.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:10.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:10.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:10.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:10.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:10.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:10.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:10.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:10.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:10.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:10.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:10.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:10.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:10.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:10.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:10.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:10.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:10.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:10.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:10.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:10.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:10.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:10.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:10.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:10.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:10.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:10.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:10.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:10.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:10.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:10.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:10.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:10.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:10.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:10.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:10.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:10.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:10.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:10.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:10.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:10.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:10.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:10.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:10.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:10.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:10.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:10.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:10.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:10.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:10.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:10.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:10.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:10.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:10.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:10.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:10.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:10.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:10.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:10.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:10.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=85, tag_tid=14, entry_tid=20581], [object_number=84, tag_tid=14, entry_tid=20580], [object_number=87, tag_tid=14, entry_tid=20579], [object_number=86, tag_tid=14, entry_tid=20578]]], state=connected] 2024-04-23T21:26:10.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:10.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:10.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:10.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:10.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:10.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:10.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:10.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:10.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:10.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:10.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:10.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:10.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:10.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:10.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:10.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=85, tag_tid=14, entry_tid=20581], [object_number=84, tag_tid=14, entry_tid=20580], [object_number=87, tag_tid=14, entry_tid=20579], [object_number=86, tag_tid=14, entry_tid=20578]]], state=connected] 2024-04-23T21:26:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:10.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=85, tag_tid=14, entry_tid=20581' 2024-04-23T21:26:10.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=85, tag_tid=14, entry_tid=20581' 2024-04-23T21:26:10.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=85, tag_tid=14, entry_tid=20581' '!=' 'object_number=86, tag_tid=14, entry_tid=20350' 2024-04-23T21:26:10.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:10.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=85, tag_tid=14, entry_tid=20581' = 'object_number=86, tag_tid=14, entry_tid=20350' 2024-04-23T21:26:10.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=85, tag_tid=14, entry_tid=20581' 2024-04-23T21:26:10.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:10.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:10.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:10.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:10.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=14, entry_tid=20581' 2024-04-23T21:26:10.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:10.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:10.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=14, entry_tid=20581' 2024-04-23T21:26:10.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:10.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=20581 2024-04-23T21:26:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 20581 2024-04-23T21:26:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:10.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:10.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:10.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:10.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:10.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:10.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:10.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:10.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:10.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:10.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:10.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:10.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:10.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:10.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:10.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:10.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:10.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:10.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:10.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:10.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:10.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:10.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:10.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:10.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:10.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:10.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:10.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:10.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:10.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:10.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:10.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:10.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:10.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:10.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:10.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:10.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:10.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:10.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:10.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:10.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:10.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:10.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:10.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:10.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:10.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:10.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:10.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:10.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:10.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:10.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:10.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:10.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:10.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:10.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:10.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:10.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:10.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:10.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:10.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:10.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:10.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:10.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:10.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:10.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:10.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:10.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:10.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:10.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:10.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=20782], [object_number=85, tag_tid=14, entry_tid=20781], [object_number=84, tag_tid=14, entry_tid=20780], [object_number=87, tag_tid=14, entry_tid=20779]]], state=connected] 2024-04-23T21:26:10.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:11.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:11.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:11.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:11.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:11.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=20782], [object_number=85, tag_tid=14, entry_tid=20781], [object_number=84, tag_tid=14, entry_tid=20780], [object_number=87, tag_tid=14, entry_tid=20779]]], state=connected] 2024-04-23T21:26:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:11.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=86, tag_tid=14, entry_tid=20782' 2024-04-23T21:26:11.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=86, tag_tid=14, entry_tid=20782' 2024-04-23T21:26:11.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=86, tag_tid=14, entry_tid=20782' '!=' 'object_number=85, tag_tid=14, entry_tid=20581' 2024-04-23T21:26:11.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:11.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=86, tag_tid=14, entry_tid=20782' = 'object_number=85, tag_tid=14, entry_tid=20581' 2024-04-23T21:26:11.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=86, tag_tid=14, entry_tid=20782' 2024-04-23T21:26:11.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:11.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:11.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:11.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:11.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=86, tag_tid=14, entry_tid=20782' 2024-04-23T21:26:11.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:11.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:11.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:11.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:11.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:11.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:11.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:11.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=86, tag_tid=14, entry_tid=20782' 2024-04-23T21:26:11.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:11.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:11.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=20782 2024-04-23T21:26:11.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 20782 2024-04-23T21:26:11.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:11.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:11.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:11.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:11.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:11.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:11.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:11.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:11.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:11.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:11.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:11.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:11.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:11.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:11.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:11.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:11.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:11.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:11.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:11.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:11.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:11.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:11.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:11.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:11.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:11.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:11.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:11.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:11.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:11.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:11.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:11.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:11.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:11.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:11.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:11.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:11.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:11.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:11.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:11.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:11.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:11.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:11.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:11.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:11.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:11.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:11.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:11.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:11.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:11.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:11.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:11.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:11.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:11.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:11.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:11.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:11.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:11.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:11.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:11.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:11.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:11.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:11.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:11.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:11.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:11.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:11.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:11.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:11.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:11.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:11.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:11.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:11.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:11.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:11.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=20918], [object_number=85, tag_tid=14, entry_tid=20917], [object_number=84, tag_tid=14, entry_tid=20916], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:11.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:11.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:11.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:11.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:11.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:11.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:11.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:11.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:11.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:11.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:11.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:11.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:11.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:11.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:11.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:11.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:11.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:11.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:11.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:11.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=20918], [object_number=85, tag_tid=14, entry_tid=20917], [object_number=84, tag_tid=14, entry_tid=20916], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:11.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:11.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:11.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:11.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=86, tag_tid=14, entry_tid=20918' '!=' 'object_number=86, tag_tid=14, entry_tid=20782' 2024-04-23T21:26:11.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:26:11.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=86, tag_tid=14, entry_tid=20918' = 'object_number=86, tag_tid=14, entry_tid=20782' 2024-04-23T21:26:11.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:11.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:11.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:11.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:11.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=15 2024-04-23T21:26:11.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:11.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:26:11.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:11.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=14 2024-04-23T21:26:11.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:11.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:11.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:11.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:26:11.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:11.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:26:11.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:26:11.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=20918 2024-04-23T21:26:11.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 15 = 14 -a 1 -le 20918 2024-04-23T21:26:11.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:26:11.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:11.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:26:11.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:11.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:11.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:11.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:11.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:11.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:11.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:11.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:11.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:11.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:11.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:11.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:11.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:11.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:11.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:11.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:11.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:11.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:11.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:11.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:11.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:11.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:11.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:11.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:11.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:11.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:11.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:11.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:11.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:11.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:11.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:11.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:11.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:11.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:11.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:11.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:11.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:11.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:11.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:11.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:11.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:11.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:11.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:11.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:11.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:11.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:11.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:11.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:11.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:11.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:11.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:11.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:11.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:11.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:12.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:12.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:12.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:12.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:12.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=20918], [object_number=85, tag_tid=14, entry_tid=20917], [object_number=84, tag_tid=14, entry_tid=20916], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:12.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:12.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:12.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:12.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:12.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:12.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:12.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=20918], [object_number=85, tag_tid=14, entry_tid=20917], [object_number=84, tag_tid=14, entry_tid=20916], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:12.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:12.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:12.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:12.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=86, tag_tid=14, entry_tid=20918' '!=' 'object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:12.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:12.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:26:12.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:12.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:12.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:12.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:12.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:12.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:12.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:12.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:12.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:12.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:12.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:12.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:12.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:12.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:12.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:12.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:12.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:12.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:12.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:12.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:12.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:12.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:12.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:12.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:12.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:12.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:12.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:12.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:12.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:12.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:12.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:12.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:12.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:12.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:12.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:12.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:12.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:12.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:12.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:12.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:12.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:12.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:12.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:12.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:12.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:12.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:12.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:12.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:12.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:12.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:12.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:12.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:12.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:12.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:12.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:12.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:12.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:12.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:12.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:12.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:12.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:12.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:12.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:12.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:12.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:12.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:12.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:12.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:12.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:12.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:12.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:12.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:12.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:12.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:12.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:12.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:12.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:12.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:12.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:12.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:12.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:12.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=20918], [object_number=85, tag_tid=14, entry_tid=20917], [object_number=84, tag_tid=14, entry_tid=20916], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:12.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:12.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:12.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:12.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:12.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:12.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:12.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:12.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:12.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:12.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:12.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=20918], [object_number=85, tag_tid=14, entry_tid=20917], [object_number=84, tag_tid=14, entry_tid=20916], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:12.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:12.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:12.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:12.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=86, tag_tid=14, entry_tid=20918' '!=' 'object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:12.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:12.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:26:13.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:13.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:13.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:13.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:13.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:13.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:13.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:13.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:13.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:13.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:13.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:13.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:13.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:13.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:13.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:13.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:13.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:13.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:13.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:13.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:13.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:13.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:13.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:13.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:13.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:13.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:13.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:13.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:13.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:13.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:13.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:13.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:13.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:13.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:13.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:13.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:13.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:13.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:13.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:13.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:13.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:13.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:13.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:13.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:13.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:13.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:13.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:13.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:13.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:13.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:13.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:13.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:13.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:13.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:13.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:13.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:13.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:13.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:13.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:13.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:13.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:13.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:13.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:13.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:13.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:13.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:13.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:13.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:13.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:13.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:13.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:13.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:13.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:13.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:13.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:13.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:13.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:13.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:13.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:13.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:13.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:13.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:13.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:13.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:13.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=20918], [object_number=85, tag_tid=14, entry_tid=20917], [object_number=84, tag_tid=14, entry_tid=20916], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:13.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:13.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:13.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:13.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:13.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:13.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:13.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:13.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:13.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:13.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:13.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:13.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:13.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:13.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:13.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:13.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:13.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:13.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:13.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:13.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:13.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:13.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=20918], [object_number=85, tag_tid=14, entry_tid=20917], [object_number=84, tag_tid=14, entry_tid=20916], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:13.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:13.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:13.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:13.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=86, tag_tid=14, entry_tid=20918' '!=' 'object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:13.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:13.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:26:15.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:15.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:15.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:15.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:15.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:15.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:15.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:15.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:15.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:15.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:15.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:15.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:15.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:15.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:15.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:15.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:15.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:15.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:15.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:15.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:15.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:15.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:15.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:15.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:15.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:15.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:15.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:15.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:15.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:15.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:15.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:15.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:15.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:15.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:15.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:15.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:15.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:15.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:15.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:15.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:15.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:15.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:15.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:15.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:15.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:15.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:15.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:15.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:15.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:15.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:15.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:15.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:15.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:15.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:15.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:15.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:15.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:15.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:15.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:15.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:15.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:15.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:15.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:15.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:15.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:15.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:15.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:15.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:15.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:15.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:15.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:15.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:15.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:15.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:15.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:15.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:15.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:15.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:15.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:15.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:15.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:15.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:15.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:15.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:15.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:15.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:15.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=20918], [object_number=85, tag_tid=14, entry_tid=20917], [object_number=84, tag_tid=14, entry_tid=20916], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:15.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:15.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:15.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:15.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:15.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:15.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:15.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:15.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:15.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:15.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:15.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:15.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:15.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:15.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:15.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=86, tag_tid=14, entry_tid=20918], [object_number=85, tag_tid=14, entry_tid=20917], [object_number=84, tag_tid=14, entry_tid=20916], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:15.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:15.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:15.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=86, tag_tid=14, entry_tid=20918' '!=' 'object_number=86, tag_tid=14, entry_tid=20918' 2024-04-23T21:26:15.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:26:15.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:26:17.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:26:17.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:17.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:17.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:17.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:26:17.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:26:17.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:26:17.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:26:17.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:26:17.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:17.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:17.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:26:17.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:26:17.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:17.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:26:17.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:26:17.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:26:17.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:17.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:26:17.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:26:17.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:17.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:17.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:26:17.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:26:17.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:17.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:17.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:17.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:26:17.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:17.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:17.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:17.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:26:17.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:17.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:26:17.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:26:17.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:17.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:17.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:26:17.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:26:17.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:17.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:17.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:17.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:26:17.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:26:17.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:17.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:17.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:26:17.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:26:17.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:26:17.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:17.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:17.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:26:17.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:26:17.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:26:17.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:26:17.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:26:17.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:26:17.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:26:17.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:26:17.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:26:17.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:26:17.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:26:17.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:26:17.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:26:17.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:26:17.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:26:17.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:26:17.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:17.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:26:17.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:17.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:26:17.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:17.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:26:17.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:26:17.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:26:17.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:26:17.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:26:17.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:26:17.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:26:17.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:26:17.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:26:17.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:26:17.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:26:17.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:26:17.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:26:17.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:26:17.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:26:17.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:26:17.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:17.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:17.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:26:17.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:26:17.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:17.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:26:17.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:17.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:17.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:17.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:26:17.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:17.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:17.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:17.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:26:17.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:17.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:17.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:17.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:17.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:17.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:18.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:18.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:18.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:18.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:18.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:18.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:18.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:18.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:26:18.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:18.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:18.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:18.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:26:18.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:26:18.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:26:18.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:26:18.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:26:18.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:26:18.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:26:18.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:26:18.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:18.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:26:18.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:18.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 21 2024-04-23T21:26:18.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 21 2024-04-23T21:26:18.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:26:18.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:18.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=85, tag_tid=15, entry_tid=1], [object_number=84, tag_tid=15, entry_tid=0], [object_number=86, tag_tid=14, entry_tid=20918], [object_number=87, tag_tid=14, entry_tid=20915]]], state=connected] 2024-04-23T21:26:18.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=85, tag_tid=15, entry_tid=1' -a 'object_number=85, tag_tid=15, entry_tid=1' = 'object_number=85, tag_tid=15, entry_tid=1' 2024-04-23T21:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror test snap5 2024-04-23T21:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap5 2024-04-23T21:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:26:18.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:26:19.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/test@snap5 2024-04-23T21:26:19.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'test': 2024-04-23T21:26:19.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:26:19.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:26:19.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 9 2024-04-23T21:26:19.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10aff12cc611 2024-04-23T21:26:19.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10aff12cc611 2024-04-23T21:26:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:26:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:26:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:26:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:26:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:20:44 2024 2024-04-23T21:26:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:25:07 2024 2024-04-23T21:26:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:25:58 2024 2024-04-23T21:26:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:26:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10aff12cc611 2024-04-23T21:26:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:26:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:26:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:26:19.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:26:19.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:19.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n snap4-clean ']' 2024-04-23T21:26:19.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap4-clean 2024-04-23T21:26:19.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:19.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:19.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap4-clean 2024-04-23T21:26:19.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:26:19.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:26:19.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:26:19.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:26:19.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:26:19.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:26:19.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:26:19.881 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:26:19.881+0000 7fc3fe09e640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:26:20.083 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:26:20.084+0000 7f32a8540640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 1 2024-04-23T21:26:20.084 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:26:20.085+0000 7f32a2d35640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 7 2024-04-23T21:26:20.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:26:20.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:26:20.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:26:20.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:20.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap5 2024-04-23T21:26:20.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:20.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:20.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap5 2024-04-23T21:26:20.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:26:20.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:26:20.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:26:20.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:26:20.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:26:21.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:26:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:26:21.463 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:26:21.463+0000 7fc3fd09c640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 5 2024-04-23T21:26:21.463 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:26:21.464+0000 7fc3f688f640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 7 2024-04-23T21:26:21.463 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:26:21.464+0000 7fc3f8893640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 7 2024-04-23T21:26:21.673 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:26:21.674+0000 7f32a4538640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 2 2024-04-23T21:26:21.816 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:26:21.817+0000 7fc3f8893640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:26:21.817 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:26:21.818+0000 7fc3fc89b640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2024-04-23T21:26:21.817 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:26:21.818+0000 7fc3f9094640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 4 2024-04-23T21:26:21.945 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:26:21.946+0000 7fc3f9094640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:26:21.945 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:26:21.946+0000 7fc3fd09c640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:26:21.946 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:26:21.947+0000 7fc3fb899640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2024-04-23T21:26:22.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:26:22.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:26:22.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:26:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ clean_snap_name=snap5-clean 2024-04-23T21:26:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap5-clean 2024-04-23T21:26:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:26:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap5-clean 2024-04-23T21:26:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap5-clean --debug-rbd=20 --debug-journaler=20 2024-04-23T21:26:22.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:26:22.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror test 2024-04-23T21:26:22.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:26:22.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:22.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:22.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:26:22.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=10 2024-04-23T21:26:22.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:26:22.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2024-04-23T21:26:24.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 54164480 bytes 2024-04-23T21:26:28.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 134108160 bytes 2024-04-23T21:26:29.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 151343104 bytes 2024-04-23T21:26:31.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 219176960 bytes 2024-04-23T21:26:32.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 261109760 bytes 2024-04-23T21:26:32.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:26:32.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:26:32.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:26:32.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:32.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2024-04-23T21:26:32.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:32.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:32.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:32.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:26:32.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:26:32.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:26:32.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:26:32.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:26:33.832 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5320699329620675) > weight (0.25) 2024-04-23T21:26:33.832 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8690590743844926) > weight (0.25) 2024-04-23T21:26:33.832 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.847040433795422) > weight (0.25) 2024-04-23T21:26:33.832 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9765680051038582) > weight (0.25) 2024-04-23T21:26:33.832 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2024-04-23T21:26:33.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2024-04-23T21:26:33.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:33.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:33.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:33.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:26:33.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:26:33.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:26:33.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:26:33.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:26:33.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2024-04-23T21:26:34.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='test: 2024-04-23T21:26:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: global_id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:26:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+replaying 2024-04-23T21:26:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: replaying, {"bytes_per_second":6104939.6,"entries_behind_primary":1,"entries_per_second":381.67,"non_primary_position":{"entry_tid":20918,"object_number":86,"tag_tid":14},"primary_position":{"entry_tid":1,"object_number":85,"tag_tid":15},"seconds_until_synced":0} 2024-04-23T21:26:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: service: mirror.1 on smithi150 2024-04-23T21:26:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:26:12 2024-04-23T21:26:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: peer_sites: 2024-04-23T21:26:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: name: cluster2 2024-04-23T21:26:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+stopped 2024-04-23T21:26:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: local image is primary 2024-04-23T21:26:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:26:12' 2024-04-23T21:26:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ state: .*up+replaying' 2024-04-23T21:26:34.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: state: up+replaying 2024-04-23T21:26:34.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ description: .*' 2024-04-23T21:26:34.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: description: replaying, {"bytes_per_second":6104939.6,"entries_behind_primary":1,"entries_per_second":381.67,"non_primary_position":{"entry_tid":20918,"object_number":86,"tag_tid":14},"primary_position":{"entry_tid":1,"object_number":85,"tag_tid":15},"seconds_until_synced":0} 2024-04-23T21:26:34.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:26:34.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:26:34.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^up+' 2024-04-23T21:26:34.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:26:34.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep 'service: *mirror..* on ' 2024-04-23T21:26:34.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: service: mirror.1 on smithi150 2024-04-23T21:26:34.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update 2024-04-23T21:26:34.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-04-23T21:26:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:26:12' 2024-04-23T21:26:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2024-04-23 21:26:12' 2024-04-23T21:26:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:26:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:26:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'prev_last_update=2024-04-23 21:26:12' 2024-04-23T21:26:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:26:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:26:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-04-23T21:26:34.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='OKOKOK142184274mirror.1smithi15019.0.0-3212-g8254cb5afalseOK42224277mirror.3smithi15019.0.0-3212-g8254cb5afalseOK43434346mirror.2smithi15019.0.0-3212-g8254cb5atrueOK43674369mirror.0smithi15019.0.0-3212-g8254cb5afalseOKteste56bd005-f8a9-476d-a8fc-713b4e8cd286up+replayingreplaying, {"bytes_per_second":6104939.6,"entries_behind_primary":1,"entries_per_second":381.67,"non_primary_position":{"entry_tid":20918,"object_number":86,"tag_tid":14},"primary_position":{"entry_tid":1,"object_number":85,"tag_tid":15},"seconds_until_synced":0}42184274mirror.1smithi1502024-04-23 21:26:12cluster267253cce-c621-4f62-a184-da88591999d7up+stoppedlocal image is primary2024-04-23 21:26:12' 2024-04-23T21:26:34.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update state 2024-04-23T21:26:34.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2024-04-23T21:26:34.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:26:12' 2024-04-23T21:26:34.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2024-04-23T21:26:34.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ state=up+replaying 2024-04-23T21:26:34.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:26:34.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep up+replaying 2024-04-23T21:26:34.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:26:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:26:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap6 2024-04-23T21:26:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap6 2024-04-23T21:26:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:26:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:26:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:26:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap6 2024-04-23T21:26:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap6 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:26:37.728 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2024-04-23T21:26:37.728 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2024-04-23T21:26:37.728 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2024-04-23T21:26:37.729 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:26:37.730+0000 7f68b3e11640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 42521) UID: 1000 2024-04-23T21:26:45.729 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2024-04-23T21:26:45.729 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:26:45.730 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2024-04-23T21:26:45.730 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2024-04-23T21:26:45.730 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2024-04-23T21:26:45.733 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2024-04-23T21:26:45.733 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2024-04-23T21:27:05.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror test 2024-04-23T21:27:05.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:05.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:05.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:05.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2024-04-23T21:27:05.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:05.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:05.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:05.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:27:05.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:27:05.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:05.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:27:06.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror test started 2024-04-23T21:27:06.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:27:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:27:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:27:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2024-04-23T21:27:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:27:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:27:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:27:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:27:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:27:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:06.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:27:06.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:27:06.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:27:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:27:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:27:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:27:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:27:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:27:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:27:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:27:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:27:06.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:27:06.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:06.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:27:06.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:27:06.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:27:06.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:27:06.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:27:06.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:27:06.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:27:06.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:27:06.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:27:06.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:06.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:06.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:27:06.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:27:06.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:27:06.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:27:06.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:27:06.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:06.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:06.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:06.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:27:06.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:27:06.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:27:06.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:27:06.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:27:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:27:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:27:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:27:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:27:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:27:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:27:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:27:06.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:27:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:27:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:27:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:27:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:27:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:27:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:27:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:27:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:27:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:27:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:27:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:27:06.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:27:06.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:27:06.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:27:06.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:06.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:06.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:06.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:27:06.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:27:06.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:27:06.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:27:06.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:27:06.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:27:06.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:27:06.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:27:06.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:06.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:27:06.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:27:06.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:27:06.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:06.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:06.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:27:06.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:27:06.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:27:06.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:27:06.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:06.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:06.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:27:06.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:27:06.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:27:06.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:27:06.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:27:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:27:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:27:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:27:06.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:27:06.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:27:06.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:27:06.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:06.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:06.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:27:06.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:27:06.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:06.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:27:06.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:27:06.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:27:06.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:06.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:27:06.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:27:06.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:06.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:06.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:27:06.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:27:06.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:06.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:06.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:06.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:27:06.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:06.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:06.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:06.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:27:06.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:06.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:27:06.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:27:06.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:06.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:06.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:27:06.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:27:06.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:06.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:06.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:06.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:06.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:06.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:06.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:06.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:27:06.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:06.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:27:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:06.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:27:06.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:27:06.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:27:06.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:27:06.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:06.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:27:06.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:27:06.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:06.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:06.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:27:06.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:27:06.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:06.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:06.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:06.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:27:06.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:06.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:06.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:27:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:27:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:27:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:27:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:27:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:06.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:06.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:27:06.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:06.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:27:06.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:06.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:27:06.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:06.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:27:06.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:06.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:06.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:06.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:27:06.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:06.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:06.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:06.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:27:06.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:06.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:06.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:06.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:06.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:06.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:07.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:07.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:07.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:07.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:07.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=101, tag_tid=17, entry_tid=14813], [object_number=100, tag_tid=17, entry_tid=14812], [object_number=103, tag_tid=17, entry_tid=14811], [object_number=102, tag_tid=17, entry_tid=14810]]], state=connected] 2024-04-23T21:27:07.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:07.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:07.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:27:07.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:07.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:07.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:07.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:27:07.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:07.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:07.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:07.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:27:07.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:07.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:07.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:07.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:07.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:07.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:07.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=101, tag_tid=17, entry_tid=14813], [object_number=100, tag_tid=17, entry_tid=14812], [object_number=103, tag_tid=17, entry_tid=14811], [object_number=102, tag_tid=17, entry_tid=14810]]], state=connected] 2024-04-23T21:27:07.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:07.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=101, tag_tid=17, entry_tid=14813' 2024-04-23T21:27:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=101, tag_tid=18, entry_tid=1' -a 'object_number=101, tag_tid=18, entry_tid=1' = 'object_number=101, tag_tid=17, entry_tid=14813' 2024-04-23T21:27:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=101, tag_tid=17, entry_tid=14813' '!=' '' 2024-04-23T21:27:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:27:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=101, tag_tid=17, entry_tid=14813' = '' 2024-04-23T21:27:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=101, tag_tid=17, entry_tid=14813' 2024-04-23T21:27:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:07.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=18 2024-04-23T21:27:07.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=17, entry_tid=14813' 2024-04-23T21:27:07.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:07.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:07.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=17 2024-04-23T21:27:07.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:07.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:07.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:07.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:27:07.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=17, entry_tid=14813' 2024-04-23T21:27:07.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:07.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=14813 2024-04-23T21:27:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 18 = 17 -a 1 -le 14813 2024-04-23T21:27:07.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:27:07.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:27:07.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:27:07.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:27:07.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:07.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:07.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:07.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:27:07.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:27:07.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:27:07.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:27:07.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:27:07.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:07.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:07.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:27:07.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:27:07.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:07.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:27:07.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:27:07.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:27:07.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:07.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:27:07.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:27:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:27:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:27:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:27:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:27:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:07.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:27:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:27:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:27:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:27:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:27:07.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:07.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:27:07.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:07.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:07.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:07.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:27:07.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:27:07.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:27:07.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:27:07.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:07.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:27:07.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:27:07.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:07.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:07.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:27:07.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:27:07.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:07.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:07.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:07.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:27:07.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:07.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:07.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:07.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:27:07.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:07.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:27:07.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:27:07.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:07.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:07.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:27:07.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:27:07.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:07.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:07.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:07.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:07.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:07.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:07.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:07.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:27:07.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:07.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:27:07.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:07.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:07.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:07.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:27:07.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:07.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:27:07.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:07.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:07.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:07.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:27:07.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:07.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:07.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:07.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:27:07.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:07.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:07.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:07.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:07.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:07.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:07.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:07.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:07.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:07.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:07.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=103, tag_tid=17, entry_tid=15103], [object_number=102, tag_tid=17, entry_tid=15102], [object_number=101, tag_tid=17, entry_tid=15101], [object_number=100, tag_tid=17, entry_tid=15100]]], state=connected] 2024-04-23T21:27:07.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:07.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:27:07.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:07.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:07.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:07.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:27:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:27:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:07.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:07.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:07.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:07.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:07.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:07.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:07.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:07.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=103, tag_tid=17, entry_tid=15103], [object_number=102, tag_tid=17, entry_tid=15102], [object_number=101, tag_tid=17, entry_tid=15101], [object_number=100, tag_tid=17, entry_tid=15100]]], state=connected] 2024-04-23T21:27:07.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:07.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=103, tag_tid=17, entry_tid=15103' 2024-04-23T21:27:07.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=101, tag_tid=18, entry_tid=1' -a 'object_number=101, tag_tid=18, entry_tid=1' = 'object_number=103, tag_tid=17, entry_tid=15103' 2024-04-23T21:27:07.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=103, tag_tid=17, entry_tid=15103' '!=' 'object_number=101, tag_tid=17, entry_tid=14813' 2024-04-23T21:27:07.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:27:07.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=103, tag_tid=17, entry_tid=15103' = 'object_number=101, tag_tid=17, entry_tid=14813' 2024-04-23T21:27:07.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=103, tag_tid=17, entry_tid=15103' 2024-04-23T21:27:07.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:07.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:07.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:07.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=18 2024-04-23T21:27:07.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=103, tag_tid=17, entry_tid=15103' 2024-04-23T21:27:07.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:07.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:07.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=17 2024-04-23T21:27:07.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:07.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:07.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:07.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:27:07.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=103, tag_tid=17, entry_tid=15103' 2024-04-23T21:27:07.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:07.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:07.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=15103 2024-04-23T21:27:07.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 18 = 17 -a 1 -le 15103 2024-04-23T21:27:07.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:27:07.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:27:07.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:27:07.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:27:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:27:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:27:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:27:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:27:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:27:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:27:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:27:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:07.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:27:07.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:27:07.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:27:07.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:07.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:27:07.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:27:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:27:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:27:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:27:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:27:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:27:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:27:07.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:07.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:07.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:27:07.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:27:07.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:07.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:07.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:07.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:07.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:07.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:07.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:07.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:27:07.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:07.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:27:07.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:07.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:07.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:07.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:27:07.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:27:07.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:27:07.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:27:07.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:07.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:27:07.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:27:07.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:27:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:27:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:27:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:27:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:27:07.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:27:07.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:07.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:07.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:27:07.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:27:07.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:07.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:07.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:07.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:07.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:07.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:07.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:07.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:27:07.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:07.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:27:07.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:07.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:07.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:07.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:27:08.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:08.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:27:08.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:08.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:08.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:08.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:27:08.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:08.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:08.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:08.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:27:08.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:08.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:08.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:08.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:08.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:08.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:08.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:08.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:08.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:08.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:08.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=100, tag_tid=17, entry_tid=15324], [object_number=103, tag_tid=17, entry_tid=15323], [object_number=102, tag_tid=17, entry_tid=15322], [object_number=101, tag_tid=17, entry_tid=15321]]], state=connected] 2024-04-23T21:27:08.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:08.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:08.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:27:08.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:08.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:08.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:08.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:27:08.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:08.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:08.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:08.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:27:08.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:08.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:08.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:08.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:08.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:08.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:08.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:08.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:08.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:08.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:08.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=100, tag_tid=17, entry_tid=15324], [object_number=103, tag_tid=17, entry_tid=15323], [object_number=102, tag_tid=17, entry_tid=15322], [object_number=101, tag_tid=17, entry_tid=15321]]], state=connected] 2024-04-23T21:27:08.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:08.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=100, tag_tid=17, entry_tid=15324' 2024-04-23T21:27:08.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=101, tag_tid=18, entry_tid=1' -a 'object_number=101, tag_tid=18, entry_tid=1' = 'object_number=100, tag_tid=17, entry_tid=15324' 2024-04-23T21:27:08.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=100, tag_tid=17, entry_tid=15324' '!=' 'object_number=103, tag_tid=17, entry_tid=15103' 2024-04-23T21:27:08.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:27:08.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=100, tag_tid=17, entry_tid=15324' = 'object_number=103, tag_tid=17, entry_tid=15103' 2024-04-23T21:27:08.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=100, tag_tid=17, entry_tid=15324' 2024-04-23T21:27:08.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:08.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:08.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:08.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=18 2024-04-23T21:27:08.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=100, tag_tid=17, entry_tid=15324' 2024-04-23T21:27:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:08.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=17 2024-04-23T21:27:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:08.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:27:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=100, tag_tid=17, entry_tid=15324' 2024-04-23T21:27:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:08.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:08.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=15324 2024-04-23T21:27:08.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 18 = 17 -a 1 -le 15324 2024-04-23T21:27:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:27:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:27:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:27:08.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:27:08.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:08.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:08.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:08.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:27:08.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:27:08.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:27:08.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:27:08.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:27:08.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:08.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:08.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:27:08.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:27:08.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:08.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:27:08.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:27:08.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:27:08.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:08.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:27:08.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:27:08.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:27:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:27:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:27:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:27:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:27:08.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:27:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:27:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:27:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:27:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:08.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:27:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:27:08.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:27:08.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:27:08.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:27:08.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:08.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:27:08.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:27:08.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:08.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:27:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:27:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:27:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:27:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:27:08.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:27:08.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:08.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:08.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:27:08.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:27:08.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:08.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:08.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:08.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:08.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:08.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:08.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:08.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:27:08.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:08.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:27:08.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:08.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:08.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:08.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:27:08.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:08.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:27:08.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:08.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:08.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:08.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:27:08.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:08.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:08.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:08.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:27:08.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:08.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:08.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:08.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:08.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:08.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:08.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:08.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:08.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:08.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:08.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=102, tag_tid=17, entry_tid=15518], [object_number=101, tag_tid=17, entry_tid=15517], [object_number=100, tag_tid=17, entry_tid=15516], [object_number=103, tag_tid=17, entry_tid=15515]]], state=connected] 2024-04-23T21:27:08.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:08.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:08.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:27:08.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:08.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:08.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:08.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:27:08.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:08.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:08.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:08.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:27:08.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:08.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:08.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:08.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:08.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:08.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:08.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:08.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:08.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:08.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:08.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=102, tag_tid=17, entry_tid=15518], [object_number=101, tag_tid=17, entry_tid=15517], [object_number=100, tag_tid=17, entry_tid=15516], [object_number=103, tag_tid=17, entry_tid=15515]]], state=connected] 2024-04-23T21:27:08.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:08.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=102, tag_tid=17, entry_tid=15518' 2024-04-23T21:27:08.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=101, tag_tid=18, entry_tid=1' -a 'object_number=101, tag_tid=18, entry_tid=1' = 'object_number=102, tag_tid=17, entry_tid=15518' 2024-04-23T21:27:08.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=102, tag_tid=17, entry_tid=15518' '!=' 'object_number=100, tag_tid=17, entry_tid=15324' 2024-04-23T21:27:08.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:27:08.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=102, tag_tid=17, entry_tid=15518' = 'object_number=100, tag_tid=17, entry_tid=15324' 2024-04-23T21:27:08.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=102, tag_tid=17, entry_tid=15518' 2024-04-23T21:27:08.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:08.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:08.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:08.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=18 2024-04-23T21:27:08.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=102, tag_tid=17, entry_tid=15518' 2024-04-23T21:27:08.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:08.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:08.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=17 2024-04-23T21:27:08.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:08.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:08.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:08.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:27:08.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=102, tag_tid=17, entry_tid=15518' 2024-04-23T21:27:08.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:08.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:08.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=15518 2024-04-23T21:27:08.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 18 = 17 -a 1 -le 15518 2024-04-23T21:27:08.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:27:08.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:27:08.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:27:08.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:27:08.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:08.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:08.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:08.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:27:08.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:27:08.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:27:08.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:27:08.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:27:08.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:08.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:08.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:27:08.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:27:08.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:08.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:27:08.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:27:08.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:27:08.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:27:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:27:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:27:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:27:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:27:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:08.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:27:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:27:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:27:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:27:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:27:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:27:08.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:08.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:27:08.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:08.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:08.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:08.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:27:08.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:27:08.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:27:08.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:27:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:27:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:27:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:08.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:27:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:27:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:27:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:27:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:27:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:27:08.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:08.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:08.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:27:08.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:27:08.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:08.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:08.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:08.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:08.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:08.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:08.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:08.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:27:08.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:08.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:27:08.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:08.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:08.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:08.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:27:09.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:09.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:27:09.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:09.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:09.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:09.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:27:09.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:09.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:09.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:09.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:27:09.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:09.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:09.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:09.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:09.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:09.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:09.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:09.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:09.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:09.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:09.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=102, tag_tid=17, entry_tid=15718], [object_number=101, tag_tid=17, entry_tid=15717], [object_number=100, tag_tid=17, entry_tid=15716], [object_number=103, tag_tid=17, entry_tid=15715]]], state=connected] 2024-04-23T21:27:09.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:09.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:09.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:27:09.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:09.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:09.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:09.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:27:09.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:09.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:09.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:09.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:27:09.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:09.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:09.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:09.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:09.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:09.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:09.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:09.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:09.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:09.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:09.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=102, tag_tid=17, entry_tid=15718], [object_number=101, tag_tid=17, entry_tid=15717], [object_number=100, tag_tid=17, entry_tid=15716], [object_number=103, tag_tid=17, entry_tid=15715]]], state=connected] 2024-04-23T21:27:09.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:09.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=102, tag_tid=17, entry_tid=15718' 2024-04-23T21:27:09.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=101, tag_tid=18, entry_tid=1' -a 'object_number=101, tag_tid=18, entry_tid=1' = 'object_number=102, tag_tid=17, entry_tid=15718' 2024-04-23T21:27:09.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=102, tag_tid=17, entry_tid=15718' '!=' 'object_number=102, tag_tid=17, entry_tid=15518' 2024-04-23T21:27:09.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:27:09.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=102, tag_tid=17, entry_tid=15718' = 'object_number=102, tag_tid=17, entry_tid=15518' 2024-04-23T21:27:09.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=102, tag_tid=17, entry_tid=15718' 2024-04-23T21:27:09.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:09.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:09.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:09.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=18 2024-04-23T21:27:09.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=102, tag_tid=17, entry_tid=15718' 2024-04-23T21:27:09.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:09.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:09.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=17 2024-04-23T21:27:09.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:09.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:09.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:09.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:27:09.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=102, tag_tid=17, entry_tid=15718' 2024-04-23T21:27:09.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:09.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:09.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=15718 2024-04-23T21:27:09.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 18 = 17 -a 1 -le 15718 2024-04-23T21:27:09.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:27:09.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:27:09.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:27:09.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:27:09.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:09.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:09.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:09.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:27:09.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:27:09.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:27:09.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:27:09.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:27:09.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:09.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:09.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:27:09.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:27:09.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:09.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:27:09.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:27:09.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:27:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:27:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:27:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:27:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:27:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:27:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:27:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:27:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:27:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:27:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:27:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:27:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:27:09.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:09.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:09.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:09.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:27:09.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:27:09.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:27:09.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:27:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:27:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:27:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:27:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:27:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:27:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:09.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:09.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:27:09.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:09.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:27:09.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:09.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:09.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:09.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:27:09.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:09.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:27:09.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:09.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:09.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:09.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:27:09.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:09.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:09.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:09.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:27:09.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:09.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:09.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:09.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:09.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:09.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:09.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:09.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:09.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:09.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:09.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=100, tag_tid=17, entry_tid=15908], [object_number=103, tag_tid=17, entry_tid=15907], [object_number=102, tag_tid=17, entry_tid=15906], [object_number=101, tag_tid=17, entry_tid=15905]]], state=connected] 2024-04-23T21:27:09.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:09.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:09.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:27:09.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:09.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:09.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:09.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:27:09.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:09.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:09.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:09.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:27:09.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:09.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:09.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:09.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:09.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:09.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:09.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:09.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:09.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:09.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:09.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=100, tag_tid=17, entry_tid=15908], [object_number=103, tag_tid=17, entry_tid=15907], [object_number=102, tag_tid=17, entry_tid=15906], [object_number=101, tag_tid=17, entry_tid=15905]]], state=connected] 2024-04-23T21:27:09.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=100, tag_tid=17, entry_tid=15908' 2024-04-23T21:27:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=101, tag_tid=18, entry_tid=1' -a 'object_number=101, tag_tid=18, entry_tid=1' = 'object_number=100, tag_tid=17, entry_tid=15908' 2024-04-23T21:27:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=100, tag_tid=17, entry_tid=15908' '!=' 'object_number=102, tag_tid=17, entry_tid=15718' 2024-04-23T21:27:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:27:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=100, tag_tid=17, entry_tid=15908' = 'object_number=102, tag_tid=17, entry_tid=15718' 2024-04-23T21:27:09.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=100, tag_tid=17, entry_tid=15908' 2024-04-23T21:27:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:09.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=18 2024-04-23T21:27:09.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=100, tag_tid=17, entry_tid=15908' 2024-04-23T21:27:09.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:09.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:09.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=17 2024-04-23T21:27:09.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:09.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:09.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:09.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:27:09.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=100, tag_tid=17, entry_tid=15908' 2024-04-23T21:27:09.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:09.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:09.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=15908 2024-04-23T21:27:09.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 18 = 17 -a 1 -le 15908 2024-04-23T21:27:09.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:27:09.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:27:09.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:27:09.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:27:09.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:09.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:09.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:09.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:27:09.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:27:09.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:27:09.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:27:09.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:27:09.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:09.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:09.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:27:09.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:27:09.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:09.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:27:09.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:27:09.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:27:09.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:09.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:27:09.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:27:09.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:09.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:09.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:27:09.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:27:09.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:09.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:09.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:09.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:27:09.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:09.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:09.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:09.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:27:09.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:09.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:27:09.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:27:09.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:09.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:09.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:27:09.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:27:09.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:09.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:09.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:09.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:09.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:09.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:09.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:09.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:27:09.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:09.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:27:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:27:09.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:27:09.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:27:09.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:27:09.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:09.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:27:09.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:27:09.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:09.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:09.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:27:09.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:27:09.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:09.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:09.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:09.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:27:09.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:09.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:09.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:09.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:27:09.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:09.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:27:09.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:27:09.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:09.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:09.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:27:09.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:27:09.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:09.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:09.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:09.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:09.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:09.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:09.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:09.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:27:09.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:09.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:27:10.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:10.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:10.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:10.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:27:10.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:10.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:27:10.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:10.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:10.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:10.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:27:10.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:10.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:10.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:10.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:27:10.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:10.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:10.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:10.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:10.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:10.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:10.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:10.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:10.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:10.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:10.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=100, tag_tid=17, entry_tid=16108], [object_number=103, tag_tid=17, entry_tid=16107], [object_number=102, tag_tid=17, entry_tid=16106], [object_number=101, tag_tid=17, entry_tid=16105]]], state=connected] 2024-04-23T21:27:10.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:10.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:10.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:27:10.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:10.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:10.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:10.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:27:10.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:10.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:10.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:10.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:27:10.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:10.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:10.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:10.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:10.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:10.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:10.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:10.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:10.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:10.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:10.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=100, tag_tid=17, entry_tid=16108], [object_number=103, tag_tid=17, entry_tid=16107], [object_number=102, tag_tid=17, entry_tid=16106], [object_number=101, tag_tid=17, entry_tid=16105]]], state=connected] 2024-04-23T21:27:10.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:10.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=100, tag_tid=17, entry_tid=16108' 2024-04-23T21:27:10.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=101, tag_tid=18, entry_tid=1' -a 'object_number=101, tag_tid=18, entry_tid=1' = 'object_number=100, tag_tid=17, entry_tid=16108' 2024-04-23T21:27:10.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=100, tag_tid=17, entry_tid=16108' '!=' 'object_number=100, tag_tid=17, entry_tid=15908' 2024-04-23T21:27:10.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:27:10.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=100, tag_tid=17, entry_tid=16108' = 'object_number=100, tag_tid=17, entry_tid=15908' 2024-04-23T21:27:10.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=100, tag_tid=17, entry_tid=16108' 2024-04-23T21:27:10.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:10.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:10.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:10.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=18 2024-04-23T21:27:10.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=100, tag_tid=17, entry_tid=16108' 2024-04-23T21:27:10.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:10.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=17 2024-04-23T21:27:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:10.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:10.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:27:10.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=100, tag_tid=17, entry_tid=16108' 2024-04-23T21:27:10.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:10.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:10.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=16108 2024-04-23T21:27:10.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 18 = 17 -a 1 -le 16108 2024-04-23T21:27:10.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:27:10.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:27:10.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:27:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:27:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:27:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:27:10.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:27:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:27:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:27:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:27:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:27:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:27:10.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:27:10.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:27:10.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:10.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:27:10.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:27:10.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:27:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:27:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:27:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:27:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:27:10.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:27:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:27:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:27:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:27:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:10.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:27:10.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:10.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:10.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:10.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:27:10.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:27:10.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:27:10.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:27:10.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:10.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:27:10.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:27:10.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:10.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:10.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:27:10.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:27:10.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:10.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:10.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:10.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:27:10.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:10.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:10.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:10.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:27:10.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:10.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:27:10.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:27:10.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:10.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:10.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:27:10.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:27:10.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:10.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:10.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:10.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:10.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:10.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:10.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:10.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:27:10.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:10.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:27:10.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:10.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:10.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:10.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:27:10.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:10.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:27:10.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:10.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:10.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:10.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:27:10.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:10.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:10.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:10.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:27:10.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:10.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:10.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:10.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:10.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:10.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:10.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:10.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:10.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:10.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:10.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262], [object_number=101, tag_tid=17, entry_tid=16261], [object_number=100, tag_tid=17, entry_tid=16260]]], state=connected] 2024-04-23T21:27:10.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:10.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:10.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:27:10.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:10.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:10.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:10.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:27:10.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:10.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:10.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:10.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:27:10.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:10.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:10.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:10.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:10.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:10.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:10.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:10.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:10.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:10.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:10.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262], [object_number=101, tag_tid=17, entry_tid=16261], [object_number=100, tag_tid=17, entry_tid=16260]]], state=connected] 2024-04-23T21:27:10.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:10.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=103, tag_tid=17, entry_tid=16263' 2024-04-23T21:27:10.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=101, tag_tid=18, entry_tid=1' -a 'object_number=101, tag_tid=18, entry_tid=1' = 'object_number=103, tag_tid=17, entry_tid=16263' 2024-04-23T21:27:10.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=103, tag_tid=17, entry_tid=16263' '!=' 'object_number=100, tag_tid=17, entry_tid=16108' 2024-04-23T21:27:10.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:27:10.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=103, tag_tid=17, entry_tid=16263' = 'object_number=100, tag_tid=17, entry_tid=16108' 2024-04-23T21:27:10.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=103, tag_tid=17, entry_tid=16263' 2024-04-23T21:27:10.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:10.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:10.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:10.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=18 2024-04-23T21:27:10.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=103, tag_tid=17, entry_tid=16263' 2024-04-23T21:27:10.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:10.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:10.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=17 2024-04-23T21:27:10.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:10.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:10.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:10.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:27:10.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=103, tag_tid=17, entry_tid=16263' 2024-04-23T21:27:10.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:10.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:10.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=16263 2024-04-23T21:27:10.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 18 = 17 -a 1 -le 16263 2024-04-23T21:27:10.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:27:10.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:27:10.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:27:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:27:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:27:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:27:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:27:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:27:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:27:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:11.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:27:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:27:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:27:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:27:11.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:27:11.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:27:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:27:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:27:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:27:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:27:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:27:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:27:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:27:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:27:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:27:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:11.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:27:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:27:11.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:11.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:11.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:11.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:27:11.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:27:11.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:27:11.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:27:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:27:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:27:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:27:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:27:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:27:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:27:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:27:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:27:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:27:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:27:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:27:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:27:11.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:11.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:11.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:11.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:27:11.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:27:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:27:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:27:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:11.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:11.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:11.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:11.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:11.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:11.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:11.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:11.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:11.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262], [object_number=101, tag_tid=17, entry_tid=16261], [object_number=100, tag_tid=17, entry_tid=16260]]], state=connected] 2024-04-23T21:27:11.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:11.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:11.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:27:11.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:11.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:11.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:11.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:27:11.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:11.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:11.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:11.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:27:11.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:11.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:11.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:11.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:11.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:11.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:11.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:11.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:11.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:11.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:11.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262], [object_number=101, tag_tid=17, entry_tid=16261], [object_number=100, tag_tid=17, entry_tid=16260]]], state=connected] 2024-04-23T21:27:11.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=103, tag_tid=17, entry_tid=16263' 2024-04-23T21:27:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=101, tag_tid=18, entry_tid=1' -a 'object_number=101, tag_tid=18, entry_tid=1' = 'object_number=103, tag_tid=17, entry_tid=16263' 2024-04-23T21:27:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=103, tag_tid=17, entry_tid=16263' '!=' 'object_number=103, tag_tid=17, entry_tid=16263' 2024-04-23T21:27:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:27:11.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:27:11.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:27:11.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:11.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:11.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:11.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:27:11.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:27:11.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:27:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:27:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:27:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:27:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:27:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:27:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:27:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:27:11.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:11.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:27:11.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:27:11.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:11.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:11.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:27:11.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:27:11.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:11.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:11.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:11.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:27:11.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:11.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:11.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:11.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:27:11.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:11.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:27:11.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:27:11.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:11.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:11.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:27:11.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:27:11.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:11.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:11.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:11.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:11.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:11.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:11.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:11.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:27:11.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:11.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:27:11.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:11.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:11.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:11.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:27:11.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:27:11.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:27:11.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:27:11.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:11.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:27:11.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:27:11.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:11.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:11.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:27:11.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:27:11.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:11.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:11.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:11.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:27:11.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:11.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:11.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:11.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:27:11.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:11.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:27:11.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:27:11.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:11.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:11.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:27:11.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:27:11.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:11.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:11.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:11.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:11.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:11.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:11.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:11.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:27:11.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:11.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:27:11.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:11.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:11.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:11.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:27:11.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:27:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:27:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:27:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:11.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:11.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:11.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:11.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:11.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:11.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:11.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:11.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:11.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:11.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:11.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:11.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:27:11.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:11.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:11.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:11.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:27:11.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:11.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:11.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:11.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:27:11.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:11.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:11.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:11.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:11.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:11.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:12.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 25 2024-04-23T21:27:12.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 25 2024-04-23T21:27:12.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:12.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:12.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=101, tag_tid=18, entry_tid=1], [object_number=100, tag_tid=18, entry_tid=0], [object_number=103, tag_tid=17, entry_tid=16263], [object_number=102, tag_tid=17, entry_tid=16262]]], state=connected] 2024-04-23T21:27:12.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:12.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:12.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=101, tag_tid=18, entry_tid=1' -a 'object_number=101, tag_tid=18, entry_tid=1' = 'object_number=101, tag_tid=18, entry_tid=1' 2024-04-23T21:27:12.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:12.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror test snap6 2024-04-23T21:27:12.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:12.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:12.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:12.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap6 2024-04-23T21:27:12.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:27:12.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:27:12.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:27:13.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/test@snap6 2024-04-23T21:27:13.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'test': 2024-04-23T21:27:13.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:27:13.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:27:13.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 11 2024-04-23T21:27:13.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10aff12cc611 2024-04-23T21:27:13.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10aff12cc611 2024-04-23T21:27:13.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:27:13.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:27:13.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:27:13.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:27:13.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:20:44 2024 2024-04-23T21:27:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:26:19 2024 2024-04-23T21:27:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:26:59 2024 2024-04-23T21:27:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:27:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10aff12cc611 2024-04-23T21:27:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:27:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:27:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:27:13.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:27:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n snap5-clean ']' 2024-04-23T21:27:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap5-clean 2024-04-23T21:27:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap5-clean 2024-04-23T21:27:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:27:13.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:27:13.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:27:13.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:27:13.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:27:13.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:27:13.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:27:13.942 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:27:13.943+0000 7fc3fc89b640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 6 2024-04-23T21:27:13.942 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:27:13.943+0000 7fc3f8893640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 6 2024-04-23T21:27:13.943 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:27:13.943+0000 7fc3fd89d640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 3 2024-04-23T21:27:14.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:27:14.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:27:14.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:27:14.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:14.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap6 2024-04-23T21:27:14.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:14.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:14.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap6 2024-04-23T21:27:14.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:27:14.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:27:14.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:27:14.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:27:14.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:27:15.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:27:15.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:27:15.443 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:27:15.443+0000 7f9201bfb640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 1 2024-04-23T21:27:15.698 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:27:15.698+0000 7f32a8540640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 3 2024-04-23T21:27:15.698 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:27:15.699+0000 7f32a4538640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 7 2024-04-23T21:27:15.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:27:16.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:27:16.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:27:16.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:16.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ clean_snap_name=snap6-clean 2024-04-23T21:27:16.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap6-clean 2024-04-23T21:27:16.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:27:16.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:16.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:16.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap6-clean 2024-04-23T21:27:16.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap6-clean --debug-rbd=20 --debug-journaler=20 2024-04-23T21:27:17.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:27:17.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror test 2024-04-23T21:27:17.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:27:17.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:17.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:17.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:27:17.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=6 2024-04-23T21:27:17.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:27:17.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2024-04-23T21:27:19.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 54405120 bytes 2024-04-23T21:27:22.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 143191040 bytes 2024-04-23T21:27:23.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:27:23.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:27:23.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:27:23.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:23.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2024-04-23T21:27:23.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:23.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:23.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:23.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:27:23.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:27:23.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:27:23.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:23.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:27:24.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2024-04-23T21:27:24.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:24.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:24.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:24.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:27:24.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:27:24.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:27:24.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:27:24.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:27:24.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2024-04-23T21:27:24.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='test: 2024-04-23T21:27:24.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: global_id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:27:24.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+replaying 2024-04-23T21:27:24.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: replaying, {"bytes_per_second":6155848.23,"entries_behind_primary":0,"entries_per_second":382.97,"non_primary_position":{"entry_tid":1,"object_number":101,"tag_tid":18},"primary_position":{"entry_tid":1,"object_number":101,"tag_tid":18}} 2024-04-23T21:27:24.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: service: mirror.1 on smithi150 2024-04-23T21:27:24.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:27:12 2024-04-23T21:27:24.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: peer_sites: 2024-04-23T21:27:24.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: name: cluster2 2024-04-23T21:27:24.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+stopped 2024-04-23T21:27:24.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: local image is primary 2024-04-23T21:27:24.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:27:12' 2024-04-23T21:27:24.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ state: .*up+replaying' 2024-04-23T21:27:24.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: state: up+replaying 2024-04-23T21:27:24.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ description: .*' 2024-04-23T21:27:24.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: description: replaying, {"bytes_per_second":6155848.23,"entries_behind_primary":0,"entries_per_second":382.97,"non_primary_position":{"entry_tid":1,"object_number":101,"tag_tid":18},"primary_position":{"entry_tid":1,"object_number":101,"tag_tid":18}} 2024-04-23T21:27:24.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:27:24.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:27:24.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^up+' 2024-04-23T21:27:24.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:27:24.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep 'service: *mirror..* on ' 2024-04-23T21:27:24.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: service: mirror.1 on smithi150 2024-04-23T21:27:24.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update 2024-04-23T21:27:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-04-23T21:27:24.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:27:12' 2024-04-23T21:27:24.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2024-04-23 21:27:12' 2024-04-23T21:27:24.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:24.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:24.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:24.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:27:24.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'prev_last_update=2024-04-23 21:27:12' 2024-04-23T21:27:24.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:27:24.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:27:24.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-04-23T21:27:24.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='OKOKOK142184274mirror.1smithi15019.0.0-3212-g8254cb5afalseOK42224277mirror.3smithi15019.0.0-3212-g8254cb5afalseOK43434346mirror.2smithi15019.0.0-3212-g8254cb5atrueOK43674369mirror.0smithi15019.0.0-3212-g8254cb5afalseOKteste56bd005-f8a9-476d-a8fc-713b4e8cd286up+replayingreplaying, {"bytes_per_second":6155848.23,"entries_behind_primary":0,"entries_per_second":382.97,"non_primary_position":{"entry_tid":1,"object_number":101,"tag_tid":18},"primary_position":{"entry_tid":1,"object_number":101,"tag_tid":18}}42184274mirror.1smithi1502024-04-23 21:27:12cluster267253cce-c621-4f62-a184-da88591999d7up+stoppedlocal image is primary2024-04-23 21:27:12' 2024-04-23T21:27:24.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update state 2024-04-23T21:27:24.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2024-04-23T21:27:24.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:27:12' 2024-04-23T21:27:24.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2024-04-23T21:27:24.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ state=up+replaying 2024-04-23T21:27:24.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:27:24.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep up+replaying 2024-04-23T21:27:24.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:27:24.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:24.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:24.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap7 2024-04-23T21:27:24.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap7 2024-04-23T21:27:24.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:27:24.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:24.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:24.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap7 2024-04-23T21:27:24.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap7 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:27:54.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror test 2024-04-23T21:27:54.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:54.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:54.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:54.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2024-04-23T21:27:54.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:54.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:54.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:54.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:27:54.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:27:54.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:54.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:27:55.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror test started 2024-04-23T21:27:55.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:55.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:55.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:55.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:27:55.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:27:55.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:27:55.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2024-04-23T21:27:55.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:27:55.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:27:55.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:27:55.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:55.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:27:55.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:27:55.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:55.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:27:55.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:27:55.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:27:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:27:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:27:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:27:55.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:27:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:27:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:27:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:27:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:27:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:27:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:27:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:27:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:27:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:27:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:27:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:27:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:27:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:27:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:27:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:27:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:27:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:27:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:27:55.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:27:55.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:55.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:55.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:55.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:27:55.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:27:55.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:27:55.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:27:55.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:55.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:27:55.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:27:55.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:27:55.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:55.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:27:55.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:27:55.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:27:55.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:55.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:55.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:55.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:55.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:27:55.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:27:55.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:27:55.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:55.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:27:55.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:27:55.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:27:55.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:27:55.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:27:55.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:27:55.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:27:55.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:27:55.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:27:55.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:55.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:55.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:27:55.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:27:55.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:27:55.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:27:55.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:27:55.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:55.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:55.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:55.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:27:55.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:27:55.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:27:55.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:27:55.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:27:55.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:27:55.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:27:55.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:27:55.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:55.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:27:55.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:27:55.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:27:55.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:55.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:55.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:27:55.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:27:55.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:27:55.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:27:55.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:55.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:55.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:27:55.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:27:55.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:27:55.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:27:55.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:27:56.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:27:56.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:56.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:56.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:56.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:27:56.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:27:56.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:27:56.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:27:56.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:27:56.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:56.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:56.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:27:56.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:27:56.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:56.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:27:56.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:27:56.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:27:56.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:56.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:27:56.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:27:56.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:56.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:56.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:27:56.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:27:56.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:56.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:56.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:56.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:27:56.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:56.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:56.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:56.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:27:56.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:56.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:27:56.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:27:56.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:56.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:56.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:27:56.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:27:56.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:56.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:56.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:56.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:56.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:56.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:56.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:56.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:27:56.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:56.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:27:56.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:56.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:56.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:56.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:27:56.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:27:56.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:27:56.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:27:56.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:56.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:27:56.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:27:56.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:27:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:27:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:27:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:27:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:27:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:27:56.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:27:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:27:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:27:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:56.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:27:56.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:56.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:56.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:56.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:27:56.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:27:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:27:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:27:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:56.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 27 2024-04-23T21:27:56.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 27 2024-04-23T21:27:56.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:56.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=109, tag_tid=21, entry_tid=1], [object_number=108, tag_tid=21, entry_tid=0], [object_number=110, tag_tid=20, entry_tid=8094], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:27:56.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=110, tag_tid=20, entry_tid=8094], [object_number=109, tag_tid=20, entry_tid=8093], [object_number=108, tag_tid=20, entry_tid=8092], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:27:56.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:56.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=109, tag_tid=21, entry_tid=1' 2024-04-23T21:27:56.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:27:56.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:56.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:56.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:56.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:27:56.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:56.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:56.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:56.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:27:56.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:56.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:56.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:56.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:56.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:56.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:56.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 27 2024-04-23T21:27:56.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 27 2024-04-23T21:27:56.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:56.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=109, tag_tid=21, entry_tid=1], [object_number=108, tag_tid=21, entry_tid=0], [object_number=110, tag_tid=20, entry_tid=8094], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:27:56.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=110, tag_tid=20, entry_tid=8094], [object_number=109, tag_tid=20, entry_tid=8093], [object_number=108, tag_tid=20, entry_tid=8092], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:27:56.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:56.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=110, tag_tid=20, entry_tid=8094' 2024-04-23T21:27:56.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=109, tag_tid=21, entry_tid=1' -a 'object_number=109, tag_tid=21, entry_tid=1' = 'object_number=110, tag_tid=20, entry_tid=8094' 2024-04-23T21:27:56.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=110, tag_tid=20, entry_tid=8094' '!=' '' 2024-04-23T21:27:56.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:27:56.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=110, tag_tid=20, entry_tid=8094' = '' 2024-04-23T21:27:56.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=110, tag_tid=20, entry_tid=8094' 2024-04-23T21:27:56.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=109, tag_tid=21, entry_tid=1' 2024-04-23T21:27:56.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:56.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:56.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=21 2024-04-23T21:27:56.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=110, tag_tid=20, entry_tid=8094' 2024-04-23T21:27:56.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:27:56.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:56.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=20 2024-04-23T21:27:56.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=109, tag_tid=21, entry_tid=1' 2024-04-23T21:27:56.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:56.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:56.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:27:56.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=110, tag_tid=20, entry_tid=8094' 2024-04-23T21:27:56.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:27:56.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:27:56.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=8094 2024-04-23T21:27:56.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 21 = 20 -a 1 -le 8094 2024-04-23T21:27:56.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:27:56.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:27:56.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:27:56.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:27:56.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:56.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:56.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:56.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:27:56.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:27:56.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:27:56.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:27:56.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:27:56.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:56.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:56.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:27:56.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:27:56.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:56.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:27:56.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:27:56.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:27:56.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:56.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:27:56.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:27:56.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:56.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:56.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:27:56.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:27:56.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:56.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:56.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:56.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:27:56.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:56.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:56.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:56.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:27:56.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:56.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:27:56.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:27:56.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:56.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:56.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:27:56.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:27:56.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:56.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:56.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:56.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:56.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:56.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:56.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:56.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:27:56.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:56.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:27:56.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:56.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:56.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:56.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:27:56.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:27:56.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:27:56.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:27:56.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:56.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:27:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:27:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:27:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:27:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:27:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:27:56.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:56.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:27:56.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:27:56.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:56.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:56.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:27:56.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:27:56.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:56.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:56.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:56.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:56.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:56.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:56.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:56.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:27:56.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:56.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:27:56.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:56.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:56.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:56.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:27:56.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:56.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:27:56.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:56.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:56.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:56.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:27:56.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:56.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:56.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:56.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:27:56.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:56.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:56.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:56.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:56.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:56.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:56.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 27 2024-04-23T21:27:56.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 27 2024-04-23T21:27:56.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:56.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=109, tag_tid=21, entry_tid=1], [object_number=108, tag_tid=21, entry_tid=0], [object_number=110, tag_tid=20, entry_tid=8094], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:27:56.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=110, tag_tid=20, entry_tid=8094], [object_number=109, tag_tid=20, entry_tid=8093], [object_number=108, tag_tid=20, entry_tid=8092], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:27:56.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:56.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=109, tag_tid=21, entry_tid=1' 2024-04-23T21:27:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:27:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:27:56.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:56.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:56.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:56.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:27:56.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:56.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:56.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:56.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:56.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:56.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:56.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 27 2024-04-23T21:27:56.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 27 2024-04-23T21:27:56.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:56.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=109, tag_tid=21, entry_tid=1], [object_number=108, tag_tid=21, entry_tid=0], [object_number=110, tag_tid=20, entry_tid=8094], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:27:56.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=110, tag_tid=20, entry_tid=8094], [object_number=109, tag_tid=20, entry_tid=8093], [object_number=108, tag_tid=20, entry_tid=8092], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:27:56.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:56.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=110, tag_tid=20, entry_tid=8094' 2024-04-23T21:27:56.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=109, tag_tid=21, entry_tid=1' -a 'object_number=109, tag_tid=21, entry_tid=1' = 'object_number=110, tag_tid=20, entry_tid=8094' 2024-04-23T21:27:56.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=110, tag_tid=20, entry_tid=8094' '!=' 'object_number=110, tag_tid=20, entry_tid=8094' 2024-04-23T21:27:56.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:27:56.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:27:57.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:27:57.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:57.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:57.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:57.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:27:57.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:27:57.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:27:57.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:27:57.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:27:57.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:57.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:57.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:27:57.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:27:57.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:57.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:27:57.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:27:57.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:27:57.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:57.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:27:57.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:27:57.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:57.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:57.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:27:57.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:27:57.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:57.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:57.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:57.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:27:57.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:57.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:57.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:57.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:27:57.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:57.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:27:57.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:27:57.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:57.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:57.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:27:57.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:27:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:27:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:57.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:27:57.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:57.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:57.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:57.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:27:57.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:27:57.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:27:57.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:27:57.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:57.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:27:57.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:27:57.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:57.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:57.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:27:57.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:27:57.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:57.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:57.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:57.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:27:57.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:57.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:57.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:57.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:27:57.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:57.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:27:57.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:27:57.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:57.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:57.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:27:57.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:27:57.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:57.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:57.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:57.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:57.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:57.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:57.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:57.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:27:57.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:57.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:27:57.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:57.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:57.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:57.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:27:57.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:57.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:27:57.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:57.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:57.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:57.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:27:57.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:57.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:57.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:57.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:27:57.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:57.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:57.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:57.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:57.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:57.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:57.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 27 2024-04-23T21:27:57.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 27 2024-04-23T21:27:57.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:57.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=109, tag_tid=21, entry_tid=1], [object_number=108, tag_tid=21, entry_tid=0], [object_number=110, tag_tid=20, entry_tid=8094], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:27:57.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=110, tag_tid=20, entry_tid=8094], [object_number=109, tag_tid=20, entry_tid=8093], [object_number=108, tag_tid=20, entry_tid=8092], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:27:57.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:57.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=109, tag_tid=21, entry_tid=1' 2024-04-23T21:27:57.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:27:57.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:57.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:57.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:57.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:27:57.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:57.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:57.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:57.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:27:57.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:57.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:57.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:57.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:57.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:57.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:57.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 27 2024-04-23T21:27:57.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 27 2024-04-23T21:27:57.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:57.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=109, tag_tid=21, entry_tid=1], [object_number=108, tag_tid=21, entry_tid=0], [object_number=110, tag_tid=20, entry_tid=8094], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:27:57.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=110, tag_tid=20, entry_tid=8094], [object_number=109, tag_tid=20, entry_tid=8093], [object_number=108, tag_tid=20, entry_tid=8092], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:27:57.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:57.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=110, tag_tid=20, entry_tid=8094' 2024-04-23T21:27:57.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=109, tag_tid=21, entry_tid=1' -a 'object_number=109, tag_tid=21, entry_tid=1' = 'object_number=110, tag_tid=20, entry_tid=8094' 2024-04-23T21:27:57.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=110, tag_tid=20, entry_tid=8094' '!=' 'object_number=110, tag_tid=20, entry_tid=8094' 2024-04-23T21:27:57.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:27:57.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:27:57.833 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2024-04-23T21:27:57.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2024-04-23T21:27:57.834 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2024-04-23T21:27:57.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:27:57.836+0000 7f595d8f3640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 44637) UID: 1000 2024-04-23T21:27:58.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:27:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:27:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:27:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:27:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:27:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:27:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:27:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:27:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:27:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:27:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:27:58.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:27:58.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:27:58.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:27:58.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:27:58.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:58.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:27:58.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:27:58.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:58.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:58.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:27:58.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:27:58.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:58.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:58.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:58.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:27:58.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:58.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:58.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:58.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:27:58.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:58.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:27:58.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:27:58.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:58.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:58.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:27:58.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:27:58.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:58.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:58.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:58.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:27:58.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:27:58.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:58.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:58.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:27:58.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:27:58.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:27:58.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:58.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:58.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:27:58.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:27:58.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2024-04-23T21:27:58.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:27:58.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:27:58.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:27:58.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:27:58.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:27:58.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:27:58.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:27:58.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:27:58.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:27:58.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:27:58.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:27:58.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:58.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:27:58.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:58.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:27:58.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:27:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:27:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:27:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:27:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:27:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:27:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:27:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:27:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:27:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:27:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:27:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:27:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:27:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:27:58.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:27:58.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:27:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:27:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:27:58.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:27:58.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:27:58.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:27:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:27:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:58.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:58.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:58.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 27 2024-04-23T21:27:58.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 27 2024-04-23T21:27:58.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:58.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=109, tag_tid=21, entry_tid=1], [object_number=108, tag_tid=21, entry_tid=0], [object_number=110, tag_tid=20, entry_tid=8094], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:27:58.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=110, tag_tid=20, entry_tid=8094], [object_number=109, tag_tid=20, entry_tid=8093], [object_number=108, tag_tid=20, entry_tid=8092], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:27:58.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:58.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=109, tag_tid=21, entry_tid=1' 2024-04-23T21:27:58.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:27:58.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:58.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:58.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:58.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:27:58.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:27:58.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:27:58.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:27:58.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:27:58.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:27:58.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:27:58.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:27:58.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:58.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:27:58.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:58.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 27 2024-04-23T21:27:58.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 27 2024-04-23T21:27:58.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:27:58.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=109, tag_tid=21, entry_tid=1], [object_number=108, tag_tid=21, entry_tid=0], [object_number=110, tag_tid=20, entry_tid=8094], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:27:58.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=110, tag_tid=20, entry_tid=8094], [object_number=109, tag_tid=20, entry_tid=8093], [object_number=108, tag_tid=20, entry_tid=8092], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:27:58.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:27:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=110, tag_tid=20, entry_tid=8094' 2024-04-23T21:27:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=109, tag_tid=21, entry_tid=1' -a 'object_number=109, tag_tid=21, entry_tid=1' = 'object_number=110, tag_tid=20, entry_tid=8094' 2024-04-23T21:27:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=110, tag_tid=20, entry_tid=8094' '!=' 'object_number=110, tag_tid=20, entry_tid=8094' 2024-04-23T21:27:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:27:58.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:28:00.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:28:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:28:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:28:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:28:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:28:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:28:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:00.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:28:00.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:28:00.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:28:00.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:28:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:28:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:28:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:28:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:28:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:28:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:28:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:28:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:28:00.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:28:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:28:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:28:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:28:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:28:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:28:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:28:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:28:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:28:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:28:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:28:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:28:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:28:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:28:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:28:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:28:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:28:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:28:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:28:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:28:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:28:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:28:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:28:00.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:00.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:00.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:00.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:28:00.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2024-04-23T21:28:00.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:28:00.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:28:00.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:28:00.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:00.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:28:00.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:28:00.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:28:00.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:28:00.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:28:00.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:28:00.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:28:00.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:28:00.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:28:00.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:00.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:28:00.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:28:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:28:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:28:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:28:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:28:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:28:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:28:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:28:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:28:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:28:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:28:00.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:28:00.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:28:00.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:28:00.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:28:00.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:28:00.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:00.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:00.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:00.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:28:00.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:00.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:28:00.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:00.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:00.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:00.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:28:00.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:00.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:00.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:00.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:28:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:28:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:28:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:28:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:28:00.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:00.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 27 2024-04-23T21:28:00.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 27 2024-04-23T21:28:00.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:28:00.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=109, tag_tid=21, entry_tid=1], [object_number=108, tag_tid=21, entry_tid=0], [object_number=110, tag_tid=20, entry_tid=8094], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:28:00.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=109, tag_tid=21, entry_tid=1], [object_number=108, tag_tid=21, entry_tid=0], [object_number=110, tag_tid=20, entry_tid=8094], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:28:00.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:00.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=109, tag_tid=21, entry_tid=1' 2024-04-23T21:28:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:28:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:28:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:28:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:28:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:28:00.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:28:00.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:00.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:28:00.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:00.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 27 2024-04-23T21:28:00.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 27 2024-04-23T21:28:00.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:28:00.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=109, tag_tid=21, entry_tid=1], [object_number=108, tag_tid=21, entry_tid=0], [object_number=110, tag_tid=20, entry_tid=8094], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:28:00.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=109, tag_tid=21, entry_tid=1], [object_number=108, tag_tid=21, entry_tid=0], [object_number=110, tag_tid=20, entry_tid=8094], [object_number=111, tag_tid=20, entry_tid=8091]]], state=connected] 2024-04-23T21:28:00.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:00.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=109, tag_tid=21, entry_tid=1' 2024-04-23T21:28:00.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=109, tag_tid=21, entry_tid=1' -a 'object_number=109, tag_tid=21, entry_tid=1' = 'object_number=109, tag_tid=21, entry_tid=1' 2024-04-23T21:28:00.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:00.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror test snap7 2024-04-23T21:28:00.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:00.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:00.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:00.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap7 2024-04-23T21:28:00.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:28:00.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:28:00.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:28:01.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/test@snap7 2024-04-23T21:28:01.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'test': 2024-04-23T21:28:01.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:28:01.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:28:01.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 13 2024-04-23T21:28:01.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10aff12cc611 2024-04-23T21:28:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10aff12cc611 2024-04-23T21:28:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:28:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:28:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:28:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:28:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:20:44 2024 2024-04-23T21:28:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:26:19 2024 2024-04-23T21:28:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:26:59 2024 2024-04-23T21:28:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:28:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10aff12cc611 2024-04-23T21:28:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:28:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:28:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:28:01.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:28:01.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:01.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n snap6-clean ']' 2024-04-23T21:28:01.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap6-clean 2024-04-23T21:28:01.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:01.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:01.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap6-clean 2024-04-23T21:28:01.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:28:01.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:28:01.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:01.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:01.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:28:02.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:28:02.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:02.551 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:28:02.551+0000 7fc3f9094640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:28:02.709 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:28:02.709+0000 7fc3fa897640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 3 2024-04-23T21:28:03.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:28:03.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:03.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:03.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:03.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap7 2024-04-23T21:28:03.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:03.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:03.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap7 2024-04-23T21:28:03.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:28:03.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:28:03.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:03.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:03.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:28:03.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:28:03.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:04.111 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:28:04.112+0000 7f9204c01640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 3 2024-04-23T21:28:04.113 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:28:04.114+0000 7f9201bfb640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 5 2024-04-23T21:28:04.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:28:04.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:04.835 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2024-04-23T21:28:04.835 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:28:04.835 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2024-04-23T21:28:04.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2024-04-23T21:28:04.836 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2024-04-23T21:28:04.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2024-04-23T21:28:04.838 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2024-04-23T21:28:04.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:05.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ clean_snap_name=snap7-clean 2024-04-23T21:28:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap7-clean 2024-04-23T21:28:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:28:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap7-clean 2024-04-23T21:28:05.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap7-clean --debug-rbd=20 --debug-journaler=20 2024-04-23T21:28:05.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:28:05.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror test 2024-04-23T21:28:05.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:28:05.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:05.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:05.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:28:05.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=5 2024-04-23T21:28:05.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:28:05.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2024-04-23T21:28:06.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 15326208 bytes 2024-04-23T21:28:08.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 54554624 bytes 2024-04-23T21:28:10.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:28:10.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:28:10.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:28:10.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:10.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2024-04-23T21:28:10.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:10.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:10.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:10.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:28:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:28:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:28:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:28:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:28:11.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2024-04-23T21:28:11.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:11.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:11.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:11.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:28:11.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:28:11.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:28:11.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:28:11.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:28:11.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2024-04-23T21:28:11.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='test: 2024-04-23T21:28:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: global_id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:28:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+replaying 2024-04-23T21:28:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: replaying, {"bytes_per_second":3603936.4,"entries_behind_primary":1764,"entries_per_second":223.97,"non_primary_position":{"entry_tid":4375,"object_number":107,"tag_tid":20},"primary_position":{"entry_tid":6139,"object_number":107,"tag_tid":20},"seconds_until_synced":7} 2024-04-23T21:28:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: service: mirror.1 on smithi150 2024-04-23T21:28:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:27:42 2024-04-23T21:28:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: peer_sites: 2024-04-23T21:28:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: name: cluster2 2024-04-23T21:28:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+stopped 2024-04-23T21:28:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: local image is primary 2024-04-23T21:28:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:27:42' 2024-04-23T21:28:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ state: .*up+replaying' 2024-04-23T21:28:11.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: state: up+replaying 2024-04-23T21:28:11.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ description: .*' 2024-04-23T21:28:11.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: description: replaying, {"bytes_per_second":3603936.4,"entries_behind_primary":1764,"entries_per_second":223.97,"non_primary_position":{"entry_tid":4375,"object_number":107,"tag_tid":20},"primary_position":{"entry_tid":6139,"object_number":107,"tag_tid":20},"seconds_until_synced":7} 2024-04-23T21:28:11.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:28:11.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:28:11.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^up+' 2024-04-23T21:28:11.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:28:11.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep 'service: *mirror..* on ' 2024-04-23T21:28:11.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: service: mirror.1 on smithi150 2024-04-23T21:28:11.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update 2024-04-23T21:28:11.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-04-23T21:28:11.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:27:42' 2024-04-23T21:28:11.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2024-04-23 21:27:42' 2024-04-23T21:28:11.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:11.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:11.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:11.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:28:11.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'prev_last_update=2024-04-23 21:27:42' 2024-04-23T21:28:11.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:28:11.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:28:11.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-04-23T21:28:12.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='OKOKOK142184274mirror.1smithi15019.0.0-3212-g8254cb5afalseOK42224277mirror.3smithi15019.0.0-3212-g8254cb5afalseOK43434346mirror.2smithi15019.0.0-3212-g8254cb5atrueOK44594462mirror.0smithi15019.0.0-3212-g8254cb5afalseOKteste56bd005-f8a9-476d-a8fc-713b4e8cd286up+replayingreplaying, {"bytes_per_second":3603936.4,"entries_behind_primary":1764,"entries_per_second":223.97,"non_primary_position":{"entry_tid":4375,"object_number":107,"tag_tid":20},"primary_position":{"entry_tid":6139,"object_number":107,"tag_tid":20},"seconds_until_synced":7}42184274mirror.1smithi1502024-04-23 21:27:42cluster267253cce-c621-4f62-a184-da88591999d7up+stoppedlocal image is primary2024-04-23 21:27:42' 2024-04-23T21:28:12.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update state 2024-04-23T21:28:12.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2024-04-23T21:28:12.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:27:42' 2024-04-23T21:28:12.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2024-04-23T21:28:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ state=up+replaying 2024-04-23T21:28:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:28:12.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep up+replaying 2024-04-23T21:28:12.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:28:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap8 2024-04-23T21:28:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap8 2024-04-23T21:28:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:28:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap8 2024-04-23T21:28:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap8 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:28:24.734 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8276309258681115) > weight (0.25) 2024-04-23T21:28:24.734 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6467959347308802) > weight (0.25) 2024-04-23T21:28:24.734 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6333984471172448) > weight (0.25) 2024-04-23T21:28:24.734 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2024-04-23T21:28:24.735 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2024-04-23T21:28:24.735 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2024-04-23T21:28:24.736 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:28:24.737+0000 7f6f0654b640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 45672) UID: 1000 2024-04-23T21:28:28.736 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2024-04-23T21:28:28.736 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:28:28.736 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2024-04-23T21:28:28.736 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2024-04-23T21:28:28.737 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2024-04-23T21:28:28.739 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2024-04-23T21:28:28.740 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2024-04-23T21:28:41.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror test 2024-04-23T21:28:41.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:41.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:41.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:41.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2024-04-23T21:28:41.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:41.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:41.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:41.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:28:41.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:28:41.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:28:41.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:28:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror test started 2024-04-23T21:28:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:28:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:28:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:28:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2024-04-23T21:28:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:28:42.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:28:42.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:28:42.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:28:42.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:28:42.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:28:42.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:28:42.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:28:42.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:28:42.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:28:42.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:28:42.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:28:42.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:28:42.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:28:42.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:28:42.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:28:42.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:28:42.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:28:42.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:28:42.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:42.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:28:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:28:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:28:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:28:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:28:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:28:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:28:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:28:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:28:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:28:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:28:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:28:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:28:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:28:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:28:42.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:28:42.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:28:42.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:28:42.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:28:42.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:28:42.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:28:42.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:28:42.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:42.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:42.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:42.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:28:42.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:28:42.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:28:42.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:28:42.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:28:42.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:28:42.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:28:42.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:28:42.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:28:42.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:28:42.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:28:42.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:28:42.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:28:42.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:42.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:28:42.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:28:42.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:28:42.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:28:42.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:28:42.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:28:42.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:28:42.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:28:42.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:28:42.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:28:42.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:28:42.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:28:42.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:28:42.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:28:42.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:28:42.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:28:42.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:28:42.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:28:42.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:28:42.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:28:42.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:28:42.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:28:42.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:42.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:42.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:42.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:28:43.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:28:43.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:28:43.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:28:43.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:28:43.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:28:43.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:28:43.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:28:43.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:43.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:28:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:28:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:28:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:28:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:28:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:28:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:28:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:28:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:28:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:28:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:28:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:28:43.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:28:43.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:43.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:43.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:43.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:28:43.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:28:43.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:28:43.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:28:43.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:28:43.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:43.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:43.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:28:43.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:28:43.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:28:43.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:28:43.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:28:43.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:28:43.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:28:43.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:28:43.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:28:43.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:43.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:28:43.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:28:43.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:28:43.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:28:43.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:28:43.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:28:43.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:28:43.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:28:43.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:28:43.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:43.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:28:43.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:28:43.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:28:43.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:28:43.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:28:43.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:28:43.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:28:43.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:28:43.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:28:43.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:28:43.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:43.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:28:43.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:28:43.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:28:43.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:28:43.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:28:43.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:28:43.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:28:43.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:43.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:43.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:43.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:28:43.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:28:43.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:28:43.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:28:43.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:28:43.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:28:43.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:28:43.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:43.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:28:43.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:28:43.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:28:43.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:28:43.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:28:43.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:28:43.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:28:43.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:28:43.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:28:43.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:43.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:28:43.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:28:43.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:28:43.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:28:43.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:28:43.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:28:43.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:28:43.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:28:43.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:28:43.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:28:43.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:43.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:28:43.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:28:43.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:28:43.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:28:43.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:28:43.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:28:43.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:28:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:43.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:28:43.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:43.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:28:43.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:43.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:43.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:43.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:28:43.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:43.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:43.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:43.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:28:43.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:28:43.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:28:43.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:28:43.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:43.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:28:43.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:43.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 28 2024-04-23T21:28:43.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 28 2024-04-23T21:28:43.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:28:43.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=113, tag_tid=24, entry_tid=1], [object_number=112, tag_tid=24, entry_tid=0], [object_number=114, tag_tid=23, entry_tid=5962], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:43.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=114, tag_tid=23, entry_tid=5962], [object_number=113, tag_tid=23, entry_tid=5961], [object_number=112, tag_tid=23, entry_tid=5960], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:43.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:43.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=113, tag_tid=24, entry_tid=1' 2024-04-23T21:28:43.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:28:43.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:43.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:43.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:43.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:28:43.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:43.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:43.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:43.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:28:43.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:28:43.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:28:43.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:28:43.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:43.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:28:43.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:43.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 28 2024-04-23T21:28:43.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 28 2024-04-23T21:28:43.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:28:43.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=113, tag_tid=24, entry_tid=1], [object_number=112, tag_tid=24, entry_tid=0], [object_number=114, tag_tid=23, entry_tid=5962], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:43.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=114, tag_tid=23, entry_tid=5962], [object_number=113, tag_tid=23, entry_tid=5961], [object_number=112, tag_tid=23, entry_tid=5960], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:43.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:43.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=114, tag_tid=23, entry_tid=5962' 2024-04-23T21:28:43.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=113, tag_tid=24, entry_tid=1' -a 'object_number=113, tag_tid=24, entry_tid=1' = 'object_number=114, tag_tid=23, entry_tid=5962' 2024-04-23T21:28:43.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=114, tag_tid=23, entry_tid=5962' '!=' '' 2024-04-23T21:28:43.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:28:43.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=114, tag_tid=23, entry_tid=5962' = '' 2024-04-23T21:28:43.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=114, tag_tid=23, entry_tid=5962' 2024-04-23T21:28:43.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=113, tag_tid=24, entry_tid=1' 2024-04-23T21:28:43.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:28:43.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:28:43.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=24 2024-04-23T21:28:43.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=114, tag_tid=23, entry_tid=5962' 2024-04-23T21:28:43.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:28:43.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:28:43.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=23 2024-04-23T21:28:43.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=113, tag_tid=24, entry_tid=1' 2024-04-23T21:28:43.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:28:43.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:28:43.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:28:43.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=114, tag_tid=23, entry_tid=5962' 2024-04-23T21:28:43.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:28:43.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:28:43.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=5962 2024-04-23T21:28:43.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 24 = 23 -a 1 -le 5962 2024-04-23T21:28:43.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:28:43.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:28:43.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:28:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:28:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:28:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:28:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:28:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:28:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:28:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:28:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:28:43.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:28:43.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:28:43.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:28:43.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:28:43.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:28:43.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:28:43.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:28:43.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:43.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:28:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:28:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:28:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:28:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:28:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:28:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:28:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:28:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:28:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:28:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:28:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:28:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:28:43.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:28:43.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:28:43.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:28:43.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:28:43.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:28:43.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:28:43.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:43.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:28:43.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:28:43.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:28:43.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:28:43.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:28:43.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:28:43.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:28:43.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:43.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:43.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:43.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:28:43.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:28:43.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:28:43.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:28:43.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:28:43.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:28:43.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:28:43.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:43.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:28:43.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:28:43.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:28:43.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:28:43.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:28:43.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:28:43.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:28:43.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:28:43.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:28:43.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:28:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:28:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:28:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:28:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:28:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:28:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:28:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:28:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:28:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:28:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:28:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:28:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:28:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:28:43.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:28:43.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:28:43.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:28:43.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:43.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:43.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:43.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:28:43.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:43.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:28:43.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:43.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:43.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:43.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:28:43.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:43.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:43.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:43.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:28:43.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:28:43.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:28:43.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:28:43.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:43.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:28:43.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:43.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 28 2024-04-23T21:28:43.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 28 2024-04-23T21:28:43.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:28:43.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=113, tag_tid=24, entry_tid=1], [object_number=112, tag_tid=24, entry_tid=0], [object_number=114, tag_tid=23, entry_tid=5962], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:43.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=114, tag_tid=23, entry_tid=5962], [object_number=113, tag_tid=23, entry_tid=5961], [object_number=112, tag_tid=23, entry_tid=5960], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:43.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:43.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=113, tag_tid=24, entry_tid=1' 2024-04-23T21:28:43.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:28:43.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:43.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:43.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:43.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:28:43.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:43.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:43.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:43.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:28:43.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:28:43.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:28:43.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:28:43.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:43.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:28:43.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:44.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 28 2024-04-23T21:28:44.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 28 2024-04-23T21:28:44.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:28:44.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=113, tag_tid=24, entry_tid=1], [object_number=112, tag_tid=24, entry_tid=0], [object_number=114, tag_tid=23, entry_tid=5962], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:44.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=114, tag_tid=23, entry_tid=5962], [object_number=113, tag_tid=23, entry_tid=5961], [object_number=112, tag_tid=23, entry_tid=5960], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:44.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=114, tag_tid=23, entry_tid=5962' 2024-04-23T21:28:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=113, tag_tid=24, entry_tid=1' -a 'object_number=113, tag_tid=24, entry_tid=1' = 'object_number=114, tag_tid=23, entry_tid=5962' 2024-04-23T21:28:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=114, tag_tid=23, entry_tid=5962' '!=' 'object_number=114, tag_tid=23, entry_tid=5962' 2024-04-23T21:28:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:28:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:28:44.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:28:44.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:44.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:44.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:44.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:28:44.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:28:44.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:28:44.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:28:44.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:28:44.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:44.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:44.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:28:44.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:28:44.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:28:44.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:28:44.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:28:44.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:28:44.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:28:44.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:28:44.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:28:44.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:44.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:28:44.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:28:44.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:28:44.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:28:44.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:28:44.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:28:44.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:28:44.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:28:44.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:28:44.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:44.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:28:44.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:28:44.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:28:44.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:28:44.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:28:44.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:28:44.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:28:44.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:28:44.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:28:44.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:28:44.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:44.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:28:44.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:28:44.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:28:44.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:28:44.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:28:44.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:28:44.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:28:44.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:44.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:44.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:44.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:28:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:28:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:28:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:28:44.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:28:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:28:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:28:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:28:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:28:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:28:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:28:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:28:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:28:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:28:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:28:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:28:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:28:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:28:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:28:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:28:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:28:44.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:44.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:44.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:44.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:28:44.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:44.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:28:44.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:44.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:44.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:44.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:28:44.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:44.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:44.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:44.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:28:44.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:28:44.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:28:44.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:28:44.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:44.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:28:44.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:44.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 28 2024-04-23T21:28:44.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 28 2024-04-23T21:28:44.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:28:44.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=113, tag_tid=24, entry_tid=1], [object_number=112, tag_tid=24, entry_tid=0], [object_number=114, tag_tid=23, entry_tid=5962], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:44.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=114, tag_tid=23, entry_tid=5962], [object_number=113, tag_tid=23, entry_tid=5961], [object_number=112, tag_tid=23, entry_tid=5960], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:44.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:44.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=113, tag_tid=24, entry_tid=1' 2024-04-23T21:28:44.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:28:44.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:44.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:44.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:44.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:28:44.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:44.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:44.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:44.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:28:44.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:28:44.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:28:44.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:28:44.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:44.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:28:44.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:44.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 28 2024-04-23T21:28:44.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 28 2024-04-23T21:28:44.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:28:44.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=113, tag_tid=24, entry_tid=1], [object_number=112, tag_tid=24, entry_tid=0], [object_number=114, tag_tid=23, entry_tid=5962], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:44.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=114, tag_tid=23, entry_tid=5962], [object_number=113, tag_tid=23, entry_tid=5961], [object_number=112, tag_tid=23, entry_tid=5960], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:44.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:44.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=114, tag_tid=23, entry_tid=5962' 2024-04-23T21:28:44.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=113, tag_tid=24, entry_tid=1' -a 'object_number=113, tag_tid=24, entry_tid=1' = 'object_number=114, tag_tid=23, entry_tid=5962' 2024-04-23T21:28:44.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=114, tag_tid=23, entry_tid=5962' '!=' 'object_number=114, tag_tid=23, entry_tid=5962' 2024-04-23T21:28:44.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:28:44.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:28:45.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:28:45.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:45.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:45.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:45.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:28:45.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:28:45.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:28:45.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:28:45.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:28:45.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:45.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:45.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:28:45.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:28:45.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:28:45.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:28:45.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:28:45.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:28:45.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:28:45.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:28:45.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:28:45.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:45.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:28:45.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:28:45.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:28:45.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:28:45.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:28:45.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:28:45.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:28:45.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:28:45.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:28:45.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:45.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:28:45.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:28:45.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:28:45.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:28:45.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:28:45.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:28:45.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:28:45.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:28:45.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:28:45.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:28:45.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:45.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:28:45.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:28:45.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:28:45.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:28:45.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:28:45.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:28:45.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:28:45.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:45.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:45.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:45.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:28:45.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:28:45.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:28:45.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:28:45.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:28:45.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:28:45.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:28:45.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:45.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:28:45.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:28:45.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:28:45.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:28:45.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:28:45.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:28:45.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:28:45.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:28:45.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:28:45.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:45.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:28:45.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:28:45.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:28:45.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:28:45.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:28:45.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:28:45.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:28:45.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:28:45.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:28:45.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:28:45.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:45.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:28:45.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:28:45.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:28:45.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:28:45.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:28:45.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:28:45.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:28:45.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:45.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:45.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:45.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:28:45.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:45.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:28:45.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:45.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:45.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:45.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:28:45.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:45.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:45.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:45.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:28:45.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:28:45.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:28:45.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:28:45.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:45.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:28:45.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:45.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 28 2024-04-23T21:28:45.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 28 2024-04-23T21:28:45.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:28:45.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=113, tag_tid=24, entry_tid=1], [object_number=112, tag_tid=24, entry_tid=0], [object_number=114, tag_tid=23, entry_tid=5962], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:45.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=114, tag_tid=23, entry_tid=5962], [object_number=113, tag_tid=23, entry_tid=5961], [object_number=112, tag_tid=23, entry_tid=5960], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:45.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=113, tag_tid=24, entry_tid=1' 2024-04-23T21:28:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:28:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:28:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:28:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:28:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:28:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:28:45.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:45.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:28:45.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:45.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 28 2024-04-23T21:28:45.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 28 2024-04-23T21:28:45.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:28:45.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=113, tag_tid=24, entry_tid=1], [object_number=112, tag_tid=24, entry_tid=0], [object_number=114, tag_tid=23, entry_tid=5962], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:45.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=114, tag_tid=23, entry_tid=5962], [object_number=113, tag_tid=23, entry_tid=5961], [object_number=112, tag_tid=23, entry_tid=5960], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:45.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:45.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=114, tag_tid=23, entry_tid=5962' 2024-04-23T21:28:45.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=113, tag_tid=24, entry_tid=1' -a 'object_number=113, tag_tid=24, entry_tid=1' = 'object_number=114, tag_tid=23, entry_tid=5962' 2024-04-23T21:28:45.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=114, tag_tid=23, entry_tid=5962' '!=' 'object_number=114, tag_tid=23, entry_tid=5962' 2024-04-23T21:28:45.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:28:45.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:28:47.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:28:47.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:47.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:47.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:47.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:28:47.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:28:47.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:28:47.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:28:47.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:28:47.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:47.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:47.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:28:47.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:28:47.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:28:47.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:28:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:28:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:28:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:28:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:28:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:28:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:28:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:28:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:28:47.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:28:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:28:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:28:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:28:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:28:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:28:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:28:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:28:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:28:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:28:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:28:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:28:47.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:28:47.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:28:47.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:28:47.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:28:47.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:47.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:28:47.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:28:47.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:28:47.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:28:47.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:28:47.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:28:47.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:28:47.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:47.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:47.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:28:47.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:28:47.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:28:47.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:28:47.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:28:47.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:28:47.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:28:47.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:28:47.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:28:47.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:28:47.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:28:47.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:28:47.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:28:47.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:28:47.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:28:47.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:28:47.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:28:47.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:28:47.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:47.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:28:47.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:28:47.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:28:47.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:28:47.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:28:47.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:28:47.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:28:47.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:28:47.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:28:47.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:28:47.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:28:47.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:28:47.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:28:47.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:28:47.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:28:47.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:28:47.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:28:47.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:28:47.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:47.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:47.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:28:47.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:28:47.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:47.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:28:47.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:47.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:47.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:47.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:28:47.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:47.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:47.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:47.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:28:47.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:28:47.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:28:47.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:28:47.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:47.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:28:47.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:47.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 28 2024-04-23T21:28:47.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 28 2024-04-23T21:28:47.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:28:47.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=113, tag_tid=24, entry_tid=1], [object_number=112, tag_tid=24, entry_tid=0], [object_number=114, tag_tid=23, entry_tid=5962], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:47.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=113, tag_tid=24, entry_tid=1], [object_number=112, tag_tid=24, entry_tid=0], [object_number=114, tag_tid=23, entry_tid=5962], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:47.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:47.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=113, tag_tid=24, entry_tid=1' 2024-04-23T21:28:47.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:28:47.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:47.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:47.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:47.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:28:47.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:28:47.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:28:47.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:28:47.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:28:47.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:28:47.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:28:47.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:28:47.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:47.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:28:47.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:47.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 28 2024-04-23T21:28:47.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 28 2024-04-23T21:28:47.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:28:47.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=113, tag_tid=24, entry_tid=1], [object_number=112, tag_tid=24, entry_tid=0], [object_number=114, tag_tid=23, entry_tid=5962], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:47.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=113, tag_tid=24, entry_tid=1], [object_number=112, tag_tid=24, entry_tid=0], [object_number=114, tag_tid=23, entry_tid=5962], [object_number=115, tag_tid=23, entry_tid=5959]]], state=connected] 2024-04-23T21:28:47.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:28:47.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=113, tag_tid=24, entry_tid=1' 2024-04-23T21:28:47.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=113, tag_tid=24, entry_tid=1' -a 'object_number=113, tag_tid=24, entry_tid=1' = 'object_number=113, tag_tid=24, entry_tid=1' 2024-04-23T21:28:47.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:47.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror test snap8 2024-04-23T21:28:47.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:28:47.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:47.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:47.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap8 2024-04-23T21:28:47.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:28:47.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:28:47.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:28:48.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/test@snap8 2024-04-23T21:28:48.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'test': 2024-04-23T21:28:48.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:28:48.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:28:48.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 15 2024-04-23T21:28:48.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10aff12cc611 2024-04-23T21:28:48.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10aff12cc611 2024-04-23T21:28:48.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:28:48.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:28:48.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:28:48.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:28:48.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:20:44 2024 2024-04-23T21:28:48.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:28:02 2024 2024-04-23T21:28:48.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:28:10 2024 2024-04-23T21:28:48.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:28:48.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10aff12cc611 2024-04-23T21:28:48.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:28:48.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:28:48.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:28:48.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:28:48.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:48.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n snap7-clean ']' 2024-04-23T21:28:48.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap7-clean 2024-04-23T21:28:48.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:48.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:48.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap7-clean 2024-04-23T21:28:48.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:28:48.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:28:48.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:48.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:48.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:28:49.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:28:49.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:49.567 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:28:49.567+0000 7f92023fc640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 4 2024-04-23T21:28:49.568 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:28:49.569+0000 7f9200bf9640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 7 2024-04-23T21:28:49.642 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:28:49.643+0000 7fc3f9895640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 9 2024-04-23T21:28:49.643 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:28:49.643+0000 7fc3fd09c640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 9 2024-04-23T21:28:49.643 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:28:49.644+0000 7fc3f9094640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 8 2024-04-23T21:28:49.855 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:28:49.855+0000 7f32a2d35640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 2 2024-04-23T21:28:50.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:28:50.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:50.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:50.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:50.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap8 2024-04-23T21:28:50.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:50.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:50.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap8 2024-04-23T21:28:50.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:28:50.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:28:50.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:50.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:50.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:28:51.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:28:51.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:51.366 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:28:51.366+0000 7fc3fd89d640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 6 2024-04-23T21:28:51.438 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:28:51.439+0000 7f32a7d3f640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 1 2024-04-23T21:28:51.794 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:28:51.795+0000 7fc3fd09c640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 3 2024-04-23T21:28:51.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:28:51.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:52.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:28:52.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:28:52.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ clean_snap_name=snap8-clean 2024-04-23T21:28:52.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap8-clean 2024-04-23T21:28:52.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:28:52.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:52.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:52.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap8-clean 2024-04-23T21:28:52.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap8-clean --debug-rbd=20 --debug-journaler=20 2024-04-23T21:28:52.279 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:28:52.279+0000 7f83e7350640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 8 2024-04-23T21:28:52.280 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:28:52.281+0000 7f83eeb5f640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 8 2024-04-23T21:28:52.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:28:52.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror test 2024-04-23T21:28:52.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:28:52.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:28:52.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:28:52.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:28:52.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=8 2024-04-23T21:28:52.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:28:52.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2024-04-23T21:28:54.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 56037376 bytes 2024-04-23T21:28:56.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 78188544 bytes 2024-04-23T21:29:00.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:29:00.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:29:00.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:29:00.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:00.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2024-04-23T21:29:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:29:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:29:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:29:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:29:01.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2024-04-23T21:29:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:29:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:29:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:29:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:29:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:29:01.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2024-04-23T21:29:01.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='test: 2024-04-23T21:29:01.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: global_id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:29:01.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+replaying 2024-04-23T21:29:01.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: replaying, {"bytes_per_second":3165788.4,"entries_behind_primary":0,"entries_per_second":198.8,"non_primary_position":{"entry_tid":5962,"object_number":114,"tag_tid":23},"primary_position":{"entry_tid":1,"object_number":109,"tag_tid":22}} 2024-04-23T21:29:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: service: mirror.1 on smithi150 2024-04-23T21:29:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:28:42 2024-04-23T21:29:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: peer_sites: 2024-04-23T21:29:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: name: cluster2 2024-04-23T21:29:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+stopped 2024-04-23T21:29:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: local image is primary 2024-04-23T21:29:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:28:42' 2024-04-23T21:29:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ state: .*up+replaying' 2024-04-23T21:29:01.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: state: up+replaying 2024-04-23T21:29:01.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ description: .*' 2024-04-23T21:29:01.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: description: replaying, {"bytes_per_second":3165788.4,"entries_behind_primary":0,"entries_per_second":198.8,"non_primary_position":{"entry_tid":5962,"object_number":114,"tag_tid":23},"primary_position":{"entry_tid":1,"object_number":109,"tag_tid":22}} 2024-04-23T21:29:01.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:29:01.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:29:01.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^up+' 2024-04-23T21:29:01.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:29:01.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep 'service: *mirror..* on ' 2024-04-23T21:29:01.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: service: mirror.1 on smithi150 2024-04-23T21:29:01.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update 2024-04-23T21:29:01.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-04-23T21:29:01.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:28:42' 2024-04-23T21:29:01.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2024-04-23 21:28:42' 2024-04-23T21:29:01.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:01.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:01.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:01.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:29:01.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'prev_last_update=2024-04-23 21:28:42' 2024-04-23T21:29:01.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:29:01.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:29:01.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-04-23T21:29:01.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='OKOKOK142184274mirror.1smithi15019.0.0-3212-g8254cb5afalseOK42224277mirror.3smithi15019.0.0-3212-g8254cb5afalseOK43434346mirror.2smithi15019.0.0-3212-g8254cb5atrueOK44594462mirror.0smithi15019.0.0-3212-g8254cb5afalseOKteste56bd005-f8a9-476d-a8fc-713b4e8cd286up+replayingreplaying, {"bytes_per_second":3165788.4,"entries_behind_primary":0,"entries_per_second":198.8,"non_primary_position":{"entry_tid":5962,"object_number":114,"tag_tid":23},"primary_position":{"entry_tid":1,"object_number":109,"tag_tid":22}}42184274mirror.1smithi1502024-04-23 21:28:42cluster267253cce-c621-4f62-a184-da88591999d7up+stoppedlocal image is primary2024-04-23 21:28:42' 2024-04-23T21:29:01.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update state 2024-04-23T21:29:01.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2024-04-23T21:29:01.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:28:42' 2024-04-23T21:29:01.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2024-04-23T21:29:01.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ state=up+replaying 2024-04-23T21:29:01.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:29:01.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep up+replaying 2024-04-23T21:29:01.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:29:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap9 2024-04-23T21:29:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap9 2024-04-23T21:29:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:29:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap9 2024-04-23T21:29:01.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap9 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:29:08.839 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4258136631160484) > weight (0.25) 2024-04-23T21:29:08.839 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2024-04-23T21:29:08.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2024-04-23T21:29:08.840 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2024-04-23T21:29:08.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:29:08.842+0000 7f8573cd7640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 35401) UID: 1000 2024-04-23T21:29:14.841 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2024-04-23T21:29:14.841 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:29:14.842 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2024-04-23T21:29:14.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2024-04-23T21:29:14.842 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2024-04-23T21:29:14.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2024-04-23T21:29:14.845 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2024-04-23T21:29:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror test 2024-04-23T21:29:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2024-04-23T21:29:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:29:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:29:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:29:32.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror test started 2024-04-23T21:29:32.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:32.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:32.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:32.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:29:32.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:29:32.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:29:32.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2024-04-23T21:29:32.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:29:32.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:29:32.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:32.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:32.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:29:32.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:29:32.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:32.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:29:32.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:32.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:29:32.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:32.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:29:32.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:29:32.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:29:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:29:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:32.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:29:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:29:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:29:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:29:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:29:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:29:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:29:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:29:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:29:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:29:32.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:32.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:32.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:32.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:32.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:29:32.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:32.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:29:32.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:29:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:29:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:29:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:29:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:29:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:29:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:29:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:29:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:29:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:29:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:32.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:29:32.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:29:32.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:29:32.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:29:32.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:32.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:32.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:32.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:32.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:29:33.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:33.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:29:33.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:33.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:33.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2024-04-23T21:29:33.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:29:33.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:33.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:33.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:33.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:33.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:29:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:29:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:29:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:29:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:29:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:29:33.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:29:33.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:29:33.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:33.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:33.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:33.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:29:33.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:29:33.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:29:33.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:29:33.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:33.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:33.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:33.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:33.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2024-04-23T21:29:33.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:33.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:29:33.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:33.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:33.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2024-04-23T21:29:33.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:29:33.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:33.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:33.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:33.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:33.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:29:33.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:33.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:33.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:33.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:29:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:29:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:29:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:29:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:29:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:29:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:29:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:33.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:33.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:29:33.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:29:33.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:29:33.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:29:33.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:33.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:33.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:33.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:33.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2024-04-23T21:29:33.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:33.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:29:33.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:33.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:33.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 1 2024-04-23T21:29:34.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:34.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:29:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:29:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:29:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:29:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:34.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:29:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:29:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:29:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:29:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:29:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:29:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:29:34.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:34.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:34.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:34.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:29:34.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:29:34.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:29:34.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:29:34.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:34.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:34.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:34.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:34.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:29:34.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:34.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:29:34.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:34.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:34.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:29:34.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:29:34.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:34.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:34.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:34.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:34.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:29:34.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:34.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:34.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:34.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:34.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:29:34.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:34.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:34.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:34.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:34.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:29:34.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:29:34.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:29:34.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:34.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:29:34.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:29:34.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:29:34.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:34.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:34.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:34.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:29:34.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:29:34.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:29:34.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:29:34.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:34.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:34.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:34.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:34.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:29:34.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:34.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:29:34.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2024-04-23T21:29:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:29:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:34.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:29:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:29:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:29:34.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:29:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:29:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:29:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:29:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:29:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:29:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:29:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:29:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:29:34.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:34.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:34.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:34.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:34.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2024-04-23T21:29:34.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:34.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:29:34.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:34.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2024-04-23T21:29:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:29:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:29:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:29:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:34.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:29:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:29:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:29:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:29:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:29:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:29:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:29:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:29:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:29:34.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:29:34.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:34.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:34.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:34.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:34.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2024-04-23T21:29:34.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:34.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:29:34.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:34.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:34.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 2 2024-04-23T21:29:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:29:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:29:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:29:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:29:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:29:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:29:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:29:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:29:36.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:36.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:29:36.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:29:36.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:29:36.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:36.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:36.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:36.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:29:36.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:29:36.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:29:36.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:29:36.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:36.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:36.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:36.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:36.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:29:36.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:36.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:29:36.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:36.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:36.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:29:36.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:29:36.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:36.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:36.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:36.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:36.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:29:36.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:36.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:36.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:36.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:36.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:29:36.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:36.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:36.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:36.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:36.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:29:36.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:29:36.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:29:36.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:36.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:29:36.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:29:36.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:29:36.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:36.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:36.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:36.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:29:36.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:29:36.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:29:36.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:29:36.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:36.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:36.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:36.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:36.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:29:36.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:36.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:29:36.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:36.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:36.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2024-04-23T21:29:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:29:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:29:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:29:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:29:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:29:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:29:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:29:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:29:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:29:36.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:36.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:36.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:36.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:29:36.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:29:36.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:29:36.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:29:36.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:36.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:36.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:36.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:36.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2024-04-23T21:29:36.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:36.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:29:36.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:36.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:36.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2024-04-23T21:29:36.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:29:36.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:36.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:36.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:36.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:36.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:29:36.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:36.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:36.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:36.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:36.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:29:36.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:36.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:36.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:36.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:36.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:29:36.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:29:36.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:29:36.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:36.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:29:36.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:29:36.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:29:36.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:36.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:36.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:36.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:29:36.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:29:36.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:29:36.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:29:36.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:36.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:36.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:36.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:36.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2024-04-23T21:29:37.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:37.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:29:37.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:37.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:37.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 4 2024-04-23T21:29:37.740 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4127236539331778) > weight (0.25) 2024-04-23T21:29:37.740 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2024-04-23T21:29:37.740 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2024-04-23T21:29:37.741 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2024-04-23T21:29:37.742 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:29:37.742+0000 7fc060b02640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 35469) UID: 1000 2024-04-23T21:29:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:29:41.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:41.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:29:41.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:29:41.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:41.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:41.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:41.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:41.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:29:41.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:41.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:41.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:41.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:41.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:29:41.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:41.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:41.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:41.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:41.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:29:41.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:29:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:29:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:29:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:29:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:29:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:29:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:29:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:29:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:29:41.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:29:41.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:41.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:29:41.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:41.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:41.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:29:41.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:29:41.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:41.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:41.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:41.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:41.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:29:41.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:41.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:41.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:41.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:41.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:29:41.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:41.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:41.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:29:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:29:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:29:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:29:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:29:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:29:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:29:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:29:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:29:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:29:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:41.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:41.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:41.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:41.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:29:41.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:41.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:29:41.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:41.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:41.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2024-04-23T21:29:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:29:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:29:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:41.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:29:41.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:41.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:41.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:41.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:41.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:29:41.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:29:41.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:29:41.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:41.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:29:41.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:29:41.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:29:41.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:41.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:41.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:41.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:29:41.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:29:41.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:29:41.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:29:41.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:41.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:41.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:41.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:41.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2024-04-23T21:29:41.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:41.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:29:41.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2024-04-23T21:29:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:29:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:29:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:29:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:29:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:29:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:29:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:29:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:29:41.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:29:41.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:29:41.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:29:41.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:41.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:41.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:29:41.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:29:41.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:29:41.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:29:41.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:41.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:41.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:41.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:41.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2024-04-23T21:29:41.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:29:41.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:29:41.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:29:41.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:29:41.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:29:41.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:29:41.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:29:41.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:41.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:29:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:29:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:29:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:29:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:29:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:29:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:29:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:29:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:29:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:41.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:41.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:41.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:41.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:41.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:41.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:41.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:41.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:41.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:41.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:41.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:41.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:41.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:41.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:41.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:41.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:41.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:41.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:41.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:41.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:41.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:41.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:41.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:41.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:41.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:41.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:41.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:41.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:41.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:41.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:41.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:41.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:41.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:41.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:41.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:41.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:41.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:41.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:41.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:41.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:41.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:41.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:41.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:41.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:41.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:41.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:41.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:41.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:41.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:41.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:41.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:41.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:41.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:41.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:41.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:41.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:41.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:41.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:41.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:41.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:41.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:41.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:41.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:41.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:41.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:41.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:41.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:41.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:41.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:41.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:41.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:41.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:41.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:41.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:41.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:41.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:41.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:41.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:41.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:41.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:41.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:41.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:41.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:41.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:41.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:41.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:41.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:41.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:41.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:41.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:41.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:41.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:41.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:41.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:41.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:41.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:41.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:41.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:41.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:41.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:41.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:42.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:42.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:42.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:42.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:42.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:42.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:42.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:42.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:42.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:42.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:42.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:42.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:42.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:42.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:42.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:42.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 29 2024-04-23T21:29:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=117, tag_tid=26, entry_tid=4029], [object_number=116, tag_tid=26, entry_tid=4028], [object_number=119, tag_tid=26, entry_tid=4027], [object_number=118, tag_tid=26, entry_tid=4026]]], state=connected] 2024-04-23T21:29:42.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:42.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:42.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:42.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:42.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:42.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:42.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:42.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:42.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:42.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:42.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:42.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:42.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:42.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:42.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 29 2024-04-23T21:29:42.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:42.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:42.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:42.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=117, tag_tid=26, entry_tid=4029], [object_number=116, tag_tid=26, entry_tid=4028], [object_number=119, tag_tid=26, entry_tid=4027], [object_number=118, tag_tid=26, entry_tid=4026]]], state=connected] 2024-04-23T21:29:42.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:42.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=117, tag_tid=26, entry_tid=4029' 2024-04-23T21:29:42.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=117, tag_tid=26, entry_tid=4029' 2024-04-23T21:29:42.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=117, tag_tid=26, entry_tid=4029' '!=' '' 2024-04-23T21:29:42.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:42.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=117, tag_tid=26, entry_tid=4029' = '' 2024-04-23T21:29:42.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=117, tag_tid=26, entry_tid=4029' 2024-04-23T21:29:42.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:42.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=117, tag_tid=26, entry_tid=4029' 2024-04-23T21:29:42.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:42.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:42.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:42.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:42.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:42.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:42.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:42.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=117, tag_tid=26, entry_tid=4029' 2024-04-23T21:29:42.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:42.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:42.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=4029 2024-04-23T21:29:42.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 4029 2024-04-23T21:29:42.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:42.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:42.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:42.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:42.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:42.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:42.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:42.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:42.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:42.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:42.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:42.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:42.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:42.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:42.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:42.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:42.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:42.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:42.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:42.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:42.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:42.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:42.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:42.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:42.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:42.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:42.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:42.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:42.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:42.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:42.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:42.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:42.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:42.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:42.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:42.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:42.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:42.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:42.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:42.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:42.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:42.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:42.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:42.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:42.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:42.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:42.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:42.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:42.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:42.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:42.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:42.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:42.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:42.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:42.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:42.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:42.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:42.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:42.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:42.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:42.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:42.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:42.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:42.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:42.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:42.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:42.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:42.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:42.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:42.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:42.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:42.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:42.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:42.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:42.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:42.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:42.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:42.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:42.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:42.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:42.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:42.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:42.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:42.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:42.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:42.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:42.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:42.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:42.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=120, tag_tid=26, entry_tid=4244], [object_number=123, tag_tid=26, entry_tid=4243], [object_number=122, tag_tid=26, entry_tid=4242], [object_number=121, tag_tid=26, entry_tid=4241]]], state=connected] 2024-04-23T21:29:42.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:42.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:42.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:42.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:42.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:42.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:42.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:42.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:42.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:42.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:42.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:42.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:42.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:42.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:42.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:42.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:42.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:42.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:42.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:42.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:42.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:42.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=120, tag_tid=26, entry_tid=4244], [object_number=123, tag_tid=26, entry_tid=4243], [object_number=122, tag_tid=26, entry_tid=4242], [object_number=121, tag_tid=26, entry_tid=4241]]], state=connected] 2024-04-23T21:29:42.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:42.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=120, tag_tid=26, entry_tid=4244' 2024-04-23T21:29:42.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=120, tag_tid=26, entry_tid=4244' 2024-04-23T21:29:42.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=120, tag_tid=26, entry_tid=4244' '!=' 'object_number=117, tag_tid=26, entry_tid=4029' 2024-04-23T21:29:42.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:42.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=120, tag_tid=26, entry_tid=4244' = 'object_number=117, tag_tid=26, entry_tid=4029' 2024-04-23T21:29:42.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=120, tag_tid=26, entry_tid=4244' 2024-04-23T21:29:42.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:42.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=120, tag_tid=26, entry_tid=4244' 2024-04-23T21:29:42.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:42.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:42.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:42.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:42.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:42.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:42.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:42.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=120, tag_tid=26, entry_tid=4244' 2024-04-23T21:29:42.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:42.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:42.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=4244 2024-04-23T21:29:42.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 4244 2024-04-23T21:29:42.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:42.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:42.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:43.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:43.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:43.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:43.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:43.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:43.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:43.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:43.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:43.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:43.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:43.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:43.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:43.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:43.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:43.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:43.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:43.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:43.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:43.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:43.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:43.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:43.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:43.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:43.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:43.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:43.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:43.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:43.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:43.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:43.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:43.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:43.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:43.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:43.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:43.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:43.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:43.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:43.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:43.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:43.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:43.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:43.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:43.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:43.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:43.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:43.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:43.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:43.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:43.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:43.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:43.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:43.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:43.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:43.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:43.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:43.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:43.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:43.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:43.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:43.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:43.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:43.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:43.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:43.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:43.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:43.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:43.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:43.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:43.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:43.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:43.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:43.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:43.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:43.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:43.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:43.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:43.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:43.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:43.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:43.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:43.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:43.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:43.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:43.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:43.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:43.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:43.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:43.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:43.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:43.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:43.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:43.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:43.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:43.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:43.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:43.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:43.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:43.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:43.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:43.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:43.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:43.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:43.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:43.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:43.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:43.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:43.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:43.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:43.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:43.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:43.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:43.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:43.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:43.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:43.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:43.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:43.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:43.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:43.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:43.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:43.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:43.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:43.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:43.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:43.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:43.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:43.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:43.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:43.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:43.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:43.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:43.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:43.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:43.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:43.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:43.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:43.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:43.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:43.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:43.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=120, tag_tid=26, entry_tid=4560], [object_number=123, tag_tid=26, entry_tid=4559], [object_number=122, tag_tid=26, entry_tid=4558], [object_number=121, tag_tid=26, entry_tid=4557]]], state=connected] 2024-04-23T21:29:43.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:43.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:43.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:43.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:43.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:43.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:43.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:43.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:43.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:43.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:43.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:43.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:43.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:43.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:43.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:43.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:43.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:43.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:43.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:43.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:43.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:43.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=120, tag_tid=26, entry_tid=4560], [object_number=123, tag_tid=26, entry_tid=4559], [object_number=122, tag_tid=26, entry_tid=4558], [object_number=121, tag_tid=26, entry_tid=4557]]], state=connected] 2024-04-23T21:29:43.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:43.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=120, tag_tid=26, entry_tid=4560' 2024-04-23T21:29:43.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=120, tag_tid=26, entry_tid=4560' 2024-04-23T21:29:43.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=120, tag_tid=26, entry_tid=4560' '!=' 'object_number=120, tag_tid=26, entry_tid=4244' 2024-04-23T21:29:43.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:43.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=120, tag_tid=26, entry_tid=4560' = 'object_number=120, tag_tid=26, entry_tid=4244' 2024-04-23T21:29:43.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=120, tag_tid=26, entry_tid=4560' 2024-04-23T21:29:43.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:43.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:43.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:43.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:43.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=120, tag_tid=26, entry_tid=4560' 2024-04-23T21:29:43.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:43.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:43.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:43.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:43.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:43.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:43.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:43.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=120, tag_tid=26, entry_tid=4560' 2024-04-23T21:29:43.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:43.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:43.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=4560 2024-04-23T21:29:43.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 4560 2024-04-23T21:29:43.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:43.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:43.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:43.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:43.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:43.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:43.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:43.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:43.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:43.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:43.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:43.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:43.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:43.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:43.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:43.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:43.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:43.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:43.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:43.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:43.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:43.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:43.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:43.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:43.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:43.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:43.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:43.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:43.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:43.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:43.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:43.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:43.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:43.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:43.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:43.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:43.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:43.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:43.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:43.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:43.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:43.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:43.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:43.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:43.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:43.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:43.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:43.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:43.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:43.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:43.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:43.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:43.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:43.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:43.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:43.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:43.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:43.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:43.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:43.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:43.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:43.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:43.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:43.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:43.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:43.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:43.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:43.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:43.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:43.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:43.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:43.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:43.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:43.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:43.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:43.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:43.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:43.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:43.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:43.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:43.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:43.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:43.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:43.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:43.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:43.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:43.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:43.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:43.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:43.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:43.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:43.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:43.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:43.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:44.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:44.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:44.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:44.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:44.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:44.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:44.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:44.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:44.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:44.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:44.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:44.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:44.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:44.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:44.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:44.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:44.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:44.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:44.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:44.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:44.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:44.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:44.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:44.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:44.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:44.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:44.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:44.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:44.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:44.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:44.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:44.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:44.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:44.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:44.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:44.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:44.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:44.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:44.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:44.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:44.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:44.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:44.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:44.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:44.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:44.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:44.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:44.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:44.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:44.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:44.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:44.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:44.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:44.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:44.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:44.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:44.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:44.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:44.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:44.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=123, tag_tid=26, entry_tid=4783], [object_number=122, tag_tid=26, entry_tid=4782], [object_number=121, tag_tid=26, entry_tid=4781], [object_number=120, tag_tid=26, entry_tid=4780]]], state=connected] 2024-04-23T21:29:44.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:44.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:44.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:44.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:44.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:44.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:44.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:44.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:44.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:44.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:44.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:44.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:44.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:44.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:44.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:44.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:44.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:44.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:44.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:44.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:44.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:44.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=123, tag_tid=26, entry_tid=4783], [object_number=122, tag_tid=26, entry_tid=4782], [object_number=121, tag_tid=26, entry_tid=4781], [object_number=120, tag_tid=26, entry_tid=4780]]], state=connected] 2024-04-23T21:29:44.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:44.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=123, tag_tid=26, entry_tid=4783' 2024-04-23T21:29:44.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=123, tag_tid=26, entry_tid=4783' 2024-04-23T21:29:44.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=123, tag_tid=26, entry_tid=4783' '!=' 'object_number=120, tag_tid=26, entry_tid=4560' 2024-04-23T21:29:44.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:44.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=123, tag_tid=26, entry_tid=4783' = 'object_number=120, tag_tid=26, entry_tid=4560' 2024-04-23T21:29:44.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=123, tag_tid=26, entry_tid=4783' 2024-04-23T21:29:44.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:44.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:44.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:44.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:44.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=123, tag_tid=26, entry_tid=4783' 2024-04-23T21:29:44.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:44.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:44.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:44.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:44.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:44.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:44.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:44.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=123, tag_tid=26, entry_tid=4783' 2024-04-23T21:29:44.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:44.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:44.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=4783 2024-04-23T21:29:44.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 4783 2024-04-23T21:29:44.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:44.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:44.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:44.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:44.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:44.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:44.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:44.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:44.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:44.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:44.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:44.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:44.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:44.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:44.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:44.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:44.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:44.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:44.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:44.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:44.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:44.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:44.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:44.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:44.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:44.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:44.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:44.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:44.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:44.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:44.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:44.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:44.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:44.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:44.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:44.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:44.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:44.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:44.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:44.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:44.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:44.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:44.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:44.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:44.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:44.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:44.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:44.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:44.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:44.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:44.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:44.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:44.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:44.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:44.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:44.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:44.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:44.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:44.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:44.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:44.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:44.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:44.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:44.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:44.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:44.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:44.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:44.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:44.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:44.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:44.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:44.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:44.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:44.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:44.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:44.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:44.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:44.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:44.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:44.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:44.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:44.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:44.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:44.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:44.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:44.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:44.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:44.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:44.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:44.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:44.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:44.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:44.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:44.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:44.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:44.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:44.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:44.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:44.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:44.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:44.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:44.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:44.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:44.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:44.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:44.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:44.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:44.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:44.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:44.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:44.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:44.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:44.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:44.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:44.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:44.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:44.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:44.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:44.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:44.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:44.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:44.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:44.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:44.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:44.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:44.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:44.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:44.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:44.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:44.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:44.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:44.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:44.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:44.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:44.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:44.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:44.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:44.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:44.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:44.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:44.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=122, tag_tid=26, entry_tid=5002], [object_number=121, tag_tid=26, entry_tid=5001], [object_number=120, tag_tid=26, entry_tid=5000], [object_number=123, tag_tid=26, entry_tid=4999]]], state=connected] 2024-04-23T21:29:44.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:44.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:44.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:44.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:44.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:44.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:44.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:44.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:44.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:44.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:44.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:44.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:44.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:44.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:44.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:44.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:44.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:44.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:44.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:44.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:44.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:44.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=122, tag_tid=26, entry_tid=5002], [object_number=121, tag_tid=26, entry_tid=5001], [object_number=120, tag_tid=26, entry_tid=5000], [object_number=123, tag_tid=26, entry_tid=4999]]], state=connected] 2024-04-23T21:29:44.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:44.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=122, tag_tid=26, entry_tid=5002' 2024-04-23T21:29:44.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=122, tag_tid=26, entry_tid=5002' 2024-04-23T21:29:44.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=122, tag_tid=26, entry_tid=5002' '!=' 'object_number=123, tag_tid=26, entry_tid=4783' 2024-04-23T21:29:44.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:44.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=122, tag_tid=26, entry_tid=5002' = 'object_number=123, tag_tid=26, entry_tid=4783' 2024-04-23T21:29:44.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=122, tag_tid=26, entry_tid=5002' 2024-04-23T21:29:44.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:44.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:44.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:44.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:44.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=122, tag_tid=26, entry_tid=5002' 2024-04-23T21:29:44.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:44.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:44.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:44.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:44.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:44.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:44.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:44.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=122, tag_tid=26, entry_tid=5002' 2024-04-23T21:29:44.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:44.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:44.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=5002 2024-04-23T21:29:44.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 5002 2024-04-23T21:29:44.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:44.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:44.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:45.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:45.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:45.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:45.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:45.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:45.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:45.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:45.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:45.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:45.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:45.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:45.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:45.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:45.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:45.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:45.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:45.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:45.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:45.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:45.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:45.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:45.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:45.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:45.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:45.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:45.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:45.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:45.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:45.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:45.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:45.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:45.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:45.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:45.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:45.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:45.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:45.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:45.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:45.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:45.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:45.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:45.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:45.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:45.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:45.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:45.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:45.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:45.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:45.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:45.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:45.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:45.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:45.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:45.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:45.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:45.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:45.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:45.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:45.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:45.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:45.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:45.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:45.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:45.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:45.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:45.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:45.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:45.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:45.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:45.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:45.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:45.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:45.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:45.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:45.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:45.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:45.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:45.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:45.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:45.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:45.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:45.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:45.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:45.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:45.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:45.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:45.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:45.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:45.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:45.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:45.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:45.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:45.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:45.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:45.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:45.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:45.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:45.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:45.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:45.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:45.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:45.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:45.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:45.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:45.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:45.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:45.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:45.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:45.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:45.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:45.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:45.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:45.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:45.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:45.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:45.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:45.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:45.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:45.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=120, tag_tid=26, entry_tid=5272], [object_number=123, tag_tid=26, entry_tid=5271], [object_number=122, tag_tid=26, entry_tid=5270], [object_number=121, tag_tid=26, entry_tid=5269]]], state=connected] 2024-04-23T21:29:45.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:45.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:45.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:45.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:45.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:45.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:45.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:45.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:45.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:45.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:45.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:45.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:45.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:45.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:45.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:45.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:45.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=120, tag_tid=26, entry_tid=5272], [object_number=123, tag_tid=26, entry_tid=5271], [object_number=122, tag_tid=26, entry_tid=5270], [object_number=121, tag_tid=26, entry_tid=5269]]], state=connected] 2024-04-23T21:29:45.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:45.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=120, tag_tid=26, entry_tid=5272' 2024-04-23T21:29:45.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=120, tag_tid=26, entry_tid=5272' 2024-04-23T21:29:45.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=120, tag_tid=26, entry_tid=5272' '!=' 'object_number=122, tag_tid=26, entry_tid=5002' 2024-04-23T21:29:45.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:45.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=120, tag_tid=26, entry_tid=5272' = 'object_number=122, tag_tid=26, entry_tid=5002' 2024-04-23T21:29:45.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=120, tag_tid=26, entry_tid=5272' 2024-04-23T21:29:45.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:45.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:45.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:45.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:45.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=120, tag_tid=26, entry_tid=5272' 2024-04-23T21:29:45.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:45.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:45.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:45.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:45.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:45.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:45.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:45.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=120, tag_tid=26, entry_tid=5272' 2024-04-23T21:29:45.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:45.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:45.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=5272 2024-04-23T21:29:45.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 5272 2024-04-23T21:29:45.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:45.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:45.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:45.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:45.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:45.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:45.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:45.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:45.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:45.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:45.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:45.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:45.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:45.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:45.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:45.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:45.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:45.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:45.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:45.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:45.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:45.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:45.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:45.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:45.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:45.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:45.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:45.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:45.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:45.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:45.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:45.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:45.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:45.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:45.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:45.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:45.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:45.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:45.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:45.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:45.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:45.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:45.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:45.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:45.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:45.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:45.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:45.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:45.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:45.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:45.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:45.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:45.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:45.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:45.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:45.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:45.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:45.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:45.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:45.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:45.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:45.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:45.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:45.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:45.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:45.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:45.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:45.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:45.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:45.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:45.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:45.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:45.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:45.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:45.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:45.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:45.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:45.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:45.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:45.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:45.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:45.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:45.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:45.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:45.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:45.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:45.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:45.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:45.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:45.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:45.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:45.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:45.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:45.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:45.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:46.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:46.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:46.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:46.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:46.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:46.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:46.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:46.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:46.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:46.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:46.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:46.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:46.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:46.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:46.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:46.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:46.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:46.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:46.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:46.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:46.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:46.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:46.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:46.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:46.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:46.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:46.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:46.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:46.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:46.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:46.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:46.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:46.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:46.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:46.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:46.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:46.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:46.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:46.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:46.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:46.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:46.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:46.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:46.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:46.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:46.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:46.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:46.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:46.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:46.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:46.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:46.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:46.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:46.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:46.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:46.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:46.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:46.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:46.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:46.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:46.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:46.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:46.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:46.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:46.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:46.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:46.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:46.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:46.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:46.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=123, tag_tid=26, entry_tid=5547], [object_number=122, tag_tid=26, entry_tid=5546], [object_number=121, tag_tid=26, entry_tid=5545], [object_number=120, tag_tid=26, entry_tid=5544]]], state=connected] 2024-04-23T21:29:46.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:46.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:46.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:46.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:46.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:46.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:46.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:46.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:46.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:46.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:46.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:46.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:46.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:46.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:46.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:46.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:46.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:46.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:46.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:46.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:46.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:46.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=123, tag_tid=26, entry_tid=5547], [object_number=122, tag_tid=26, entry_tid=5546], [object_number=121, tag_tid=26, entry_tid=5545], [object_number=120, tag_tid=26, entry_tid=5544]]], state=connected] 2024-04-23T21:29:46.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=123, tag_tid=26, entry_tid=5547' 2024-04-23T21:29:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=123, tag_tid=26, entry_tid=5547' 2024-04-23T21:29:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=123, tag_tid=26, entry_tid=5547' '!=' 'object_number=120, tag_tid=26, entry_tid=5272' 2024-04-23T21:29:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=123, tag_tid=26, entry_tid=5547' = 'object_number=120, tag_tid=26, entry_tid=5272' 2024-04-23T21:29:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=123, tag_tid=26, entry_tid=5547' 2024-04-23T21:29:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=123, tag_tid=26, entry_tid=5547' 2024-04-23T21:29:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:46.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:46.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:46.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:46.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:46.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:46.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=123, tag_tid=26, entry_tid=5547' 2024-04-23T21:29:46.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:46.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:46.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=5547 2024-04-23T21:29:46.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 5547 2024-04-23T21:29:46.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:46.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:46.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:46.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:46.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:46.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:46.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:46.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:46.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:46.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:46.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:46.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:46.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:46.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:46.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:46.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:46.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:46.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:46.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:46.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:46.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:46.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:46.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:46.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:46.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:46.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:46.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:46.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:46.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:46.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:46.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:46.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:46.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:46.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:46.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:46.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:46.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:46.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:46.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:46.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:46.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:46.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:46.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:46.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:46.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:46.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:46.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:46.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:46.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:46.741 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2024-04-23T21:29:46.741 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:29:46.742 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2024-04-23T21:29:46.742 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2024-04-23T21:29:46.742 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2024-04-23T21:29:46.745 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2024-04-23T21:29:46.745 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2024-04-23T21:29:46.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:46.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:46.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:46.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:46.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:46.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:46.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:46.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:46.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:46.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:46.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:46.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:46.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:46.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:46.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:46.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:46.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:46.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:46.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:46.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:46.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:46.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:46.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:46.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:46.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:46.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:46.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:46.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:46.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:46.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:46.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:46.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:46.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:46.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:46.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:47.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:47.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:47.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:47.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:47.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:47.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:47.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:47.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:47.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:47.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:47.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:47.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:47.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:47.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:47.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:47.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:47.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:47.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:47.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:47.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:47.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=120, tag_tid=26, entry_tid=5840], [object_number=123, tag_tid=26, entry_tid=5839], [object_number=122, tag_tid=26, entry_tid=5838], [object_number=121, tag_tid=26, entry_tid=5837]]], state=connected] 2024-04-23T21:29:47.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:47.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:47.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:47.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:47.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:47.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:47.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:47.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:47.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:47.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:47.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:47.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:47.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:47.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:47.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:47.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:47.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:47.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:47.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:47.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:47.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:47.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=120, tag_tid=26, entry_tid=5840], [object_number=123, tag_tid=26, entry_tid=5839], [object_number=122, tag_tid=26, entry_tid=5838], [object_number=121, tag_tid=26, entry_tid=5837]]], state=connected] 2024-04-23T21:29:47.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:47.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=120, tag_tid=26, entry_tid=5840' 2024-04-23T21:29:47.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=120, tag_tid=26, entry_tid=5840' 2024-04-23T21:29:47.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=120, tag_tid=26, entry_tid=5840' '!=' 'object_number=123, tag_tid=26, entry_tid=5547' 2024-04-23T21:29:47.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:47.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=120, tag_tid=26, entry_tid=5840' = 'object_number=123, tag_tid=26, entry_tid=5547' 2024-04-23T21:29:47.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=120, tag_tid=26, entry_tid=5840' 2024-04-23T21:29:47.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:47.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:47.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:47.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:47.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=120, tag_tid=26, entry_tid=5840' 2024-04-23T21:29:47.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:47.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:47.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:47.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:47.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:47.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:47.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:47.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=120, tag_tid=26, entry_tid=5840' 2024-04-23T21:29:47.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:47.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:47.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=5840 2024-04-23T21:29:47.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 5840 2024-04-23T21:29:47.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:47.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:47.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:47.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:47.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:47.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:47.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:47.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:47.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:47.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:47.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:47.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:47.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:47.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:47.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:47.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:47.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:47.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:47.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:47.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:47.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:47.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:47.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:47.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:47.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:47.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:47.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:47.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:47.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:47.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:47.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:47.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:47.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:47.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:47.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:47.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:47.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:47.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:47.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:47.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:47.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:47.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:47.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:47.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:47.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:47.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:47.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:47.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:47.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:47.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:47.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:47.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:47.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:47.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:47.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:47.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:47.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:47.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:47.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:47.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:47.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:47.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:47.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:47.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:47.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:47.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:47.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:47.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:47.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:47.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:47.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:47.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:47.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:47.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:47.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:47.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:47.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:47.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:47.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:47.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:47.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:47.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:47.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:47.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:47.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:47.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:47.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:47.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:47.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:47.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:47.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:47.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:47.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:47.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:47.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:47.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:47.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:47.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:47.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:47.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:47.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:47.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:47.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:47.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:47.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:47.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:47.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:47.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:47.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:47.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:47.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:47.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:47.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:47.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:47.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:47.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:47.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:47.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:47.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:47.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:47.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:47.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:47.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:47.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:47.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:47.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:47.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:47.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:47.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:47.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:47.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:47.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:47.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:47.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:47.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:47.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:47.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:47.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:47.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:47.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=123, tag_tid=26, entry_tid=6123], [object_number=122, tag_tid=26, entry_tid=6122], [object_number=121, tag_tid=26, entry_tid=6121], [object_number=120, tag_tid=26, entry_tid=6120]]], state=connected] 2024-04-23T21:29:47.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:47.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:47.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:47.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:47.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:47.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:47.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:47.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:47.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:47.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:47.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:47.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:47.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:47.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:47.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:47.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:47.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:48.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:48.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:48.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:48.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:48.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=123, tag_tid=26, entry_tid=6123], [object_number=122, tag_tid=26, entry_tid=6122], [object_number=121, tag_tid=26, entry_tid=6121], [object_number=120, tag_tid=26, entry_tid=6120]]], state=connected] 2024-04-23T21:29:48.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:48.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=123, tag_tid=26, entry_tid=6123' 2024-04-23T21:29:48.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=123, tag_tid=26, entry_tid=6123' 2024-04-23T21:29:48.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=123, tag_tid=26, entry_tid=6123' '!=' 'object_number=120, tag_tid=26, entry_tid=5840' 2024-04-23T21:29:48.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:48.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=123, tag_tid=26, entry_tid=6123' = 'object_number=120, tag_tid=26, entry_tid=5840' 2024-04-23T21:29:48.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=123, tag_tid=26, entry_tid=6123' 2024-04-23T21:29:48.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:48.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:48.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:48.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:48.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=123, tag_tid=26, entry_tid=6123' 2024-04-23T21:29:48.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:48.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:48.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:48.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:48.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:48.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:48.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:48.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=123, tag_tid=26, entry_tid=6123' 2024-04-23T21:29:48.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:48.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=6123 2024-04-23T21:29:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 6123 2024-04-23T21:29:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:48.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:48.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:48.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:48.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:48.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:48.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:48.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:48.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:48.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:48.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:48.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:48.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:48.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:48.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:48.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:48.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:48.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:48.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:48.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:48.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:48.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:48.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:48.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:48.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:48.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:48.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:48.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:48.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:48.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:48.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:48.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:48.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:48.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:48.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:48.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:48.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:48.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:48.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:48.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:48.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:48.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:48.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:48.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:48.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:48.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:48.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:48.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:48.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:48.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:48.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:48.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:48.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:48.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:48.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:48.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:48.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:48.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:48.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:48.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:48.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:48.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:48.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:48.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:48.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:48.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:48.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:48.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:48.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:48.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:48.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:48.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:48.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:48.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:48.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:48.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:48.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:48.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:48.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:48.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:48.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:48.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:48.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:48.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:48.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:48.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:48.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:48.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:48.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:48.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:48.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:48.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:48.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:48.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:48.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:48.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:48.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:48.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:48.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:48.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:48.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:48.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:48.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:48.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:48.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:48.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:48.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:48.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:48.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:48.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:48.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:48.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:48.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:48.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:48.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:48.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:48.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=121, tag_tid=26, entry_tid=6509], [object_number=120, tag_tid=26, entry_tid=6508], [object_number=123, tag_tid=26, entry_tid=6507], [object_number=122, tag_tid=26, entry_tid=6506]]], state=connected] 2024-04-23T21:29:48.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:48.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:48.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:48.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:48.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:48.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:48.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:48.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:48.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:48.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:48.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:48.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:48.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:48.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:48.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:48.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:48.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:48.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:48.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:48.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:48.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:48.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=121, tag_tid=26, entry_tid=6509], [object_number=120, tag_tid=26, entry_tid=6508], [object_number=123, tag_tid=26, entry_tid=6507], [object_number=122, tag_tid=26, entry_tid=6506]]], state=connected] 2024-04-23T21:29:48.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:48.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=121, tag_tid=26, entry_tid=6509' 2024-04-23T21:29:48.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=121, tag_tid=26, entry_tid=6509' 2024-04-23T21:29:48.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=121, tag_tid=26, entry_tid=6509' '!=' 'object_number=123, tag_tid=26, entry_tid=6123' 2024-04-23T21:29:48.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:48.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=121, tag_tid=26, entry_tid=6509' = 'object_number=123, tag_tid=26, entry_tid=6123' 2024-04-23T21:29:48.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=121, tag_tid=26, entry_tid=6509' 2024-04-23T21:29:48.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:48.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:48.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:48.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:48.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=121, tag_tid=26, entry_tid=6509' 2024-04-23T21:29:48.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:48.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:48.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:48.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:48.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:48.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:48.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:48.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=121, tag_tid=26, entry_tid=6509' 2024-04-23T21:29:48.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:48.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:48.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=6509 2024-04-23T21:29:48.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 6509 2024-04-23T21:29:48.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:48.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:48.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:48.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:48.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:48.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:48.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:48.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:48.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:48.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:48.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:48.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:48.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:48.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:48.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:48.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:48.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:48.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:48.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:48.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:48.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:48.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:48.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:48.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:48.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:48.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:48.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:48.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:48.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:48.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:48.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:48.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:48.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:48.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:48.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:48.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:49.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:49.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:49.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:49.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:49.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:49.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:49.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:49.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:49.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:49.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:49.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:49.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:49.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:49.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:49.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:49.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:49.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:49.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:49.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:49.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:49.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:49.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:49.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:49.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:49.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:49.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:49.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:49.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:49.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:49.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:49.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:49.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:49.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:49.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:49.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:49.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:49.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:49.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:49.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:49.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:49.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:49.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:49.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:49.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:49.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:49.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:49.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:49.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:49.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:49.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:49.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:49.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:49.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:49.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:49.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:49.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:49.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:49.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:49.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:49.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:49.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:49.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:49.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:49.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:49.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:49.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:49.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:49.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:49.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:49.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:49.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:49.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:49.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:49.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:49.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:49.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:49.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:49.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:49.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:49.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:49.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:49.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:49.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:49.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:49.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=123, tag_tid=26, entry_tid=6787], [object_number=122, tag_tid=26, entry_tid=6786], [object_number=121, tag_tid=26, entry_tid=6785], [object_number=120, tag_tid=26, entry_tid=6784]]], state=connected] 2024-04-23T21:29:49.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:49.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:49.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:49.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:49.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:49.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:49.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:49.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:49.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:49.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:49.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:49.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:49.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:49.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:49.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:49.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:49.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:49.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:49.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:49.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:49.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:49.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=123, tag_tid=26, entry_tid=6787], [object_number=122, tag_tid=26, entry_tid=6786], [object_number=121, tag_tid=26, entry_tid=6785], [object_number=120, tag_tid=26, entry_tid=6784]]], state=connected] 2024-04-23T21:29:49.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:49.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=123, tag_tid=26, entry_tid=6787' 2024-04-23T21:29:49.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=123, tag_tid=26, entry_tid=6787' 2024-04-23T21:29:49.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=123, tag_tid=26, entry_tid=6787' '!=' 'object_number=121, tag_tid=26, entry_tid=6509' 2024-04-23T21:29:49.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:49.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=123, tag_tid=26, entry_tid=6787' = 'object_number=121, tag_tid=26, entry_tid=6509' 2024-04-23T21:29:49.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=123, tag_tid=26, entry_tid=6787' 2024-04-23T21:29:49.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:49.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:49.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:49.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:49.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=123, tag_tid=26, entry_tid=6787' 2024-04-23T21:29:49.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:49.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:49.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:49.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:49.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:49.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:49.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:49.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=123, tag_tid=26, entry_tid=6787' 2024-04-23T21:29:49.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:49.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:49.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=6787 2024-04-23T21:29:49.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 6787 2024-04-23T21:29:49.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:49.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:49.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:49.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:49.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:49.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:49.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:49.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:49.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:49.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:49.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:49.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:49.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:49.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:49.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:49.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:49.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:49.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:49.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:49.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:49.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:49.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:49.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:49.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:49.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:49.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:49.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:49.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:49.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:49.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:49.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:49.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:49.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:49.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:49.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:49.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:49.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:49.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:49.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:49.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:49.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:49.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:49.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:49.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:49.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:49.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:49.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:49.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:49.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:49.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:49.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:49.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:49.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:49.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:49.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:49.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:49.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:49.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:49.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:49.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:49.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:49.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:49.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:49.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:49.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:49.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:49.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:49.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:49.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:49.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:49.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:49.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:49.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:49.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:49.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:49.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:49.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:49.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:49.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:49.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:49.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:49.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:49.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:49.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:49.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:49.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:49.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:49.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:49.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:49.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:49.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:49.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:49.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:49.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:49.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:49.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:50.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:50.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:50.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:50.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:50.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:50.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:50.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:50.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:50.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:50.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:50.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:50.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:50.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:50.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:50.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:50.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:50.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:50.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:50.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:50.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:50.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=120, tag_tid=26, entry_tid=7072], [object_number=123, tag_tid=26, entry_tid=7071], [object_number=122, tag_tid=26, entry_tid=7070], [object_number=121, tag_tid=26, entry_tid=7069]]], state=connected] 2024-04-23T21:29:50.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:50.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:50.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:50.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:50.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:50.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:50.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:50.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:50.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:50.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:50.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:50.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:50.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:50.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:50.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:50.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:50.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:50.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:50.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:50.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:50.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:50.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=120, tag_tid=26, entry_tid=7072], [object_number=123, tag_tid=26, entry_tid=7071], [object_number=122, tag_tid=26, entry_tid=7070], [object_number=121, tag_tid=26, entry_tid=7069]]], state=connected] 2024-04-23T21:29:50.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:50.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=120, tag_tid=26, entry_tid=7072' 2024-04-23T21:29:50.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=120, tag_tid=26, entry_tid=7072' 2024-04-23T21:29:50.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=120, tag_tid=26, entry_tid=7072' '!=' 'object_number=123, tag_tid=26, entry_tid=6787' 2024-04-23T21:29:50.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:50.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=120, tag_tid=26, entry_tid=7072' = 'object_number=123, tag_tid=26, entry_tid=6787' 2024-04-23T21:29:50.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=120, tag_tid=26, entry_tid=7072' 2024-04-23T21:29:50.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:50.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:50.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:50.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:50.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=120, tag_tid=26, entry_tid=7072' 2024-04-23T21:29:50.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:50.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:50.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:50.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:50.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:50.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:50.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:50.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=120, tag_tid=26, entry_tid=7072' 2024-04-23T21:29:50.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:50.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:50.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=7072 2024-04-23T21:29:50.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 7072 2024-04-23T21:29:50.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:50.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:50.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:50.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:50.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:50.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:50.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:50.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:50.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:50.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:50.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:50.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:50.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:50.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:50.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:50.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:50.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:50.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:50.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:50.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:50.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:50.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:50.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:50.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:50.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:50.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:50.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:50.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:50.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:50.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:50.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:50.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:50.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:50.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:50.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:50.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:50.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:50.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:50.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:50.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:50.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:50.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:50.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:50.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:50.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:50.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:50.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:50.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:50.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:50.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:50.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:50.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:50.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:50.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:50.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:50.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:50.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:50.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:50.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:50.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:50.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:50.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:50.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:50.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:50.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:50.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:50.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:50.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:50.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:50.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:50.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:50.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:50.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:50.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:50.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:50.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:50.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:50.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:50.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:50.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:50.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:50.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:50.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:50.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:50.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:50.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:50.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:50.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:50.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:50.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:50.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:50.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:50.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:50.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:50.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:50.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:50.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=123, tag_tid=26, entry_tid=7355], [object_number=122, tag_tid=26, entry_tid=7354], [object_number=121, tag_tid=26, entry_tid=7353], [object_number=120, tag_tid=26, entry_tid=7352]]], state=connected] 2024-04-23T21:29:50.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:50.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:50.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:50.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:50.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:50.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:50.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:50.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:50.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:50.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:50.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:50.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:50.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:50.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:50.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:50.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:50.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:50.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:50.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:50.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:50.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:50.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=123, tag_tid=26, entry_tid=7355], [object_number=122, tag_tid=26, entry_tid=7354], [object_number=121, tag_tid=26, entry_tid=7353], [object_number=120, tag_tid=26, entry_tid=7352]]], state=connected] 2024-04-23T21:29:50.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:50.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=123, tag_tid=26, entry_tid=7355' 2024-04-23T21:29:50.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=123, tag_tid=26, entry_tid=7355' 2024-04-23T21:29:50.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=123, tag_tid=26, entry_tid=7355' '!=' 'object_number=120, tag_tid=26, entry_tid=7072' 2024-04-23T21:29:50.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:50.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=123, tag_tid=26, entry_tid=7355' = 'object_number=120, tag_tid=26, entry_tid=7072' 2024-04-23T21:29:50.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=123, tag_tid=26, entry_tid=7355' 2024-04-23T21:29:50.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:50.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:50.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:50.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:50.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=123, tag_tid=26, entry_tid=7355' 2024-04-23T21:29:50.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:50.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:50.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:50.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:50.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:50.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:50.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:50.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=123, tag_tid=26, entry_tid=7355' 2024-04-23T21:29:50.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:50.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:50.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=7355 2024-04-23T21:29:50.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 7355 2024-04-23T21:29:50.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:50.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:50.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:51.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:51.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:51.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:51.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:51.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:51.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:51.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:51.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:51.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:51.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:51.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:51.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:51.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:51.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:51.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:51.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:51.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:51.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:51.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:51.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:51.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:51.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:51.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:51.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:51.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:51.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:51.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:51.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:51.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:51.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:51.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:51.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:51.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:51.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:51.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:51.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:51.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:51.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:51.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:51.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:51.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:51.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:51.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:51.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:51.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:51.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:51.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:51.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:51.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:51.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:51.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:51.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:51.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:51.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:51.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:51.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:51.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:51.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:51.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:51.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:51.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:51.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:51.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:51.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:51.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:51.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:51.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:51.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:51.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:51.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:51.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:51.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:51.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:51.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:51.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:51.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:51.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:51.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:51.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:51.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:51.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:51.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:51.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:51.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:51.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:51.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:51.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:51.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:51.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:51.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:51.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:51.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:51.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:51.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:51.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:51.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:51.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:51.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:51.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:51.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:51.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:51.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:51.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:51.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:51.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:51.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:51.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:51.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:51.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:51.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:51.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:51.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:51.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:51.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:51.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:51.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:51.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:51.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:51.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:51.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:51.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:51.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:51.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:51.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=121, tag_tid=26, entry_tid=7641], [object_number=120, tag_tid=26, entry_tid=7640], [object_number=123, tag_tid=26, entry_tid=7639], [object_number=122, tag_tid=26, entry_tid=7638]]], state=connected] 2024-04-23T21:29:51.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:51.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:51.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:51.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:51.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:51.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:51.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:51.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:51.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:51.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:51.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:51.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:51.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:51.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:51.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:51.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:51.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:51.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:51.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:51.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:51.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:51.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=121, tag_tid=26, entry_tid=7641], [object_number=120, tag_tid=26, entry_tid=7640], [object_number=123, tag_tid=26, entry_tid=7639], [object_number=122, tag_tid=26, entry_tid=7638]]], state=connected] 2024-04-23T21:29:51.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:51.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=121, tag_tid=26, entry_tid=7641' 2024-04-23T21:29:51.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=121, tag_tid=26, entry_tid=7641' 2024-04-23T21:29:51.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=121, tag_tid=26, entry_tid=7641' '!=' 'object_number=123, tag_tid=26, entry_tid=7355' 2024-04-23T21:29:51.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:51.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=121, tag_tid=26, entry_tid=7641' = 'object_number=123, tag_tid=26, entry_tid=7355' 2024-04-23T21:29:51.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=121, tag_tid=26, entry_tid=7641' 2024-04-23T21:29:51.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:51.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:51.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:51.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:51.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=121, tag_tid=26, entry_tid=7641' 2024-04-23T21:29:51.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:51.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:51.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:51.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:51.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:51.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:51.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:51.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=121, tag_tid=26, entry_tid=7641' 2024-04-23T21:29:51.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:51.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=7641 2024-04-23T21:29:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 7641 2024-04-23T21:29:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:51.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:51.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:51.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:51.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:51.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:51.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:51.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:51.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:51.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:51.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:51.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:51.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:51.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:51.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:51.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:51.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:51.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:51.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:51.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:51.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:51.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:51.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:51.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:51.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:51.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:51.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:51.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:51.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:51.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:51.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:51.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:51.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:51.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:51.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:51.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:51.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:51.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:51.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:51.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:51.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:51.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:51.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:51.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:51.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:51.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:51.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:51.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:51.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:51.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:51.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:51.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:51.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:51.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:51.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:51.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:51.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:51.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:51.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:51.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:51.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:51.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:51.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:51.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:51.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:51.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:51.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:51.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:51.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:51.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:51.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:51.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:51.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:51.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:51.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:51.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:51.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:51.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:51.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:51.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:51.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:51.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:51.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:51.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:51.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:51.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:51.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:51.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:51.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:51.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:51.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:51.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:51.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:51.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:51.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:51.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:51.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:52.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:52.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:52.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:52.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:52.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:52.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:52.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:52.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:52.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:52.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:52.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:52.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:52.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:52.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:52.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:52.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:52.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:52.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:52.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:52.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:52.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:52.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:52.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:52.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:52.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:52.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:52.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:52.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:52.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:52.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:52.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:52.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:52.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:52.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:52.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:52.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:52.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:52.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:52.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:52.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:52.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:52.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:52.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=121, tag_tid=26, entry_tid=7921], [object_number=120, tag_tid=26, entry_tid=7920], [object_number=123, tag_tid=26, entry_tid=7919], [object_number=122, tag_tid=26, entry_tid=7918]]], state=connected] 2024-04-23T21:29:52.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:52.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:52.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:52.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:52.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:52.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:52.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:52.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:52.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:52.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:52.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=121, tag_tid=26, entry_tid=7921], [object_number=120, tag_tid=26, entry_tid=7920], [object_number=123, tag_tid=26, entry_tid=7919], [object_number=122, tag_tid=26, entry_tid=7918]]], state=connected] 2024-04-23T21:29:52.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:52.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=121, tag_tid=26, entry_tid=7921' 2024-04-23T21:29:52.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=121, tag_tid=26, entry_tid=7921' 2024-04-23T21:29:52.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=121, tag_tid=26, entry_tid=7921' '!=' 'object_number=121, tag_tid=26, entry_tid=7641' 2024-04-23T21:29:52.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:52.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=121, tag_tid=26, entry_tid=7921' = 'object_number=121, tag_tid=26, entry_tid=7641' 2024-04-23T21:29:52.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=121, tag_tid=26, entry_tid=7921' 2024-04-23T21:29:52.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:52.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:52.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:52.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:52.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=121, tag_tid=26, entry_tid=7921' 2024-04-23T21:29:52.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:52.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:52.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:52.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:52.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:52.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:52.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:52.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=121, tag_tid=26, entry_tid=7921' 2024-04-23T21:29:52.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:52.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:52.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=7921 2024-04-23T21:29:52.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 7921 2024-04-23T21:29:52.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:52.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:52.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:52.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:52.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:52.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:52.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:52.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:52.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:52.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:52.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:52.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:52.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:52.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:52.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:52.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:52.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:52.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:52.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:52.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:52.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:52.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:52.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:52.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:52.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:52.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:52.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:52.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:52.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:52.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:52.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:52.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:52.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:52.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:52.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:52.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:52.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:52.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:52.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:52.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:52.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:52.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:52.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:52.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:52.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:52.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:52.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:52.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:52.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:52.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:52.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:52.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:52.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:52.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:52.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:52.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:52.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:52.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:52.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:52.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:52.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:52.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:52.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:52.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:52.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:52.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:52.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:52.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:52.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:52.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:52.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:52.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:52.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:52.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:52.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:52.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:52.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:52.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:52.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:52.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:52.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:52.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:52.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:52.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:52.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:52.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:52.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:52.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:52.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:52.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:52.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:52.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:52.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:52.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:52.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:52.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:52.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:52.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:52.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:52.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:52.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:52.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:52.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:52.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:52.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:52.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:52.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:52.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:52.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:52.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:52.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:52.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:52.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:52.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:52.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:52.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:52.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:52.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:52.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:52.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:52.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:52.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:52.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:52.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:52.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:52.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:52.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=121, tag_tid=26, entry_tid=8181], [object_number=120, tag_tid=26, entry_tid=8180], [object_number=123, tag_tid=26, entry_tid=8179], [object_number=122, tag_tid=26, entry_tid=8178]]], state=connected] 2024-04-23T21:29:52.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:52.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:52.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:52.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:52.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:52.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:52.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:52.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:52.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:52.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:52.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:52.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:52.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:52.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:52.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:52.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:52.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 30 2024-04-23T21:29:52.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:52.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:52.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:52.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=121, tag_tid=26, entry_tid=8181], [object_number=120, tag_tid=26, entry_tid=8180], [object_number=123, tag_tid=26, entry_tid=8179], [object_number=122, tag_tid=26, entry_tid=8178]]], state=connected] 2024-04-23T21:29:52.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:52.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=121, tag_tid=26, entry_tid=8181' 2024-04-23T21:29:52.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=121, tag_tid=26, entry_tid=8181' 2024-04-23T21:29:52.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=121, tag_tid=26, entry_tid=8181' '!=' 'object_number=121, tag_tid=26, entry_tid=7921' 2024-04-23T21:29:52.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:52.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=121, tag_tid=26, entry_tid=8181' = 'object_number=121, tag_tid=26, entry_tid=7921' 2024-04-23T21:29:52.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=121, tag_tid=26, entry_tid=8181' 2024-04-23T21:29:52.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:52.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:52.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:52.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:52.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=121, tag_tid=26, entry_tid=8181' 2024-04-23T21:29:52.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:52.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:52.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:52.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:52.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:52.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:52.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:52.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=121, tag_tid=26, entry_tid=8181' 2024-04-23T21:29:52.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:52.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:52.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=8181 2024-04-23T21:29:52.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 8181 2024-04-23T21:29:52.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:52.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:52.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:53.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:53.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:53.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:53.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:53.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:53.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:53.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:53.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:53.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:53.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:53.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:53.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:53.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:53.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:53.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:53.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:53.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:53.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:53.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:53.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:53.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:53.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:53.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:53.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:53.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:53.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:53.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:53.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:53.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:53.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:53.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:53.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:53.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:53.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:53.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:53.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:53.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:53.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:53.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:53.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:53.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:53.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:53.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:53.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:53.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:53.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:53.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:53.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:53.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:53.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:53.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:53.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:53.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:53.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:53.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:53.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:53.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:53.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:53.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:53.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:53.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:53.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:53.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:53.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:53.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:53.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:53.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:53.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:53.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:53.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:53.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:53.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:53.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:53.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:53.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:53.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:53.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:53.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:53.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:53.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:53.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:53.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:53.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:53.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:53.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:53.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:53.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:53.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:53.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:53.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:53.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:53.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:53.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:53.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:53.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:53.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:53.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:53.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:53.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:53.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:53.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:53.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:53.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:53.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:53.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:53.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:53.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:53.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:53.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:53.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:53.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:53.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:53.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:53.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:53.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:53.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:53.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:53.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=126, tag_tid=26, entry_tid=8402], [object_number=125, tag_tid=26, entry_tid=8401], [object_number=124, tag_tid=26, entry_tid=8400], [object_number=127, tag_tid=26, entry_tid=8399]]], state=connected] 2024-04-23T21:29:53.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:53.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:53.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:53.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:53.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:53.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:53.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:53.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:53.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:53.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:53.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:53.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:53.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:53.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:53.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:53.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:53.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:53.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:53.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:53.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:53.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:53.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=126, tag_tid=26, entry_tid=8402], [object_number=125, tag_tid=26, entry_tid=8401], [object_number=124, tag_tid=26, entry_tid=8400], [object_number=127, tag_tid=26, entry_tid=8399]]], state=connected] 2024-04-23T21:29:53.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:53.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=126, tag_tid=26, entry_tid=8402' 2024-04-23T21:29:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=126, tag_tid=26, entry_tid=8402' 2024-04-23T21:29:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=126, tag_tid=26, entry_tid=8402' '!=' 'object_number=121, tag_tid=26, entry_tid=8181' 2024-04-23T21:29:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=126, tag_tid=26, entry_tid=8402' = 'object_number=121, tag_tid=26, entry_tid=8181' 2024-04-23T21:29:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=126, tag_tid=26, entry_tid=8402' 2024-04-23T21:29:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:53.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=126, tag_tid=26, entry_tid=8402' 2024-04-23T21:29:53.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:53.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:53.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:53.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:53.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:53.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:53.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:53.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=126, tag_tid=26, entry_tid=8402' 2024-04-23T21:29:53.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:53.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:53.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=8402 2024-04-23T21:29:53.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 8402 2024-04-23T21:29:53.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:53.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:53.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:53.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:53.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:53.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:53.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:53.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:53.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:53.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:53.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:53.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:53.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:53.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:53.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:53.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:53.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:53.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:53.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:53.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:53.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:53.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:53.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:53.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:53.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:53.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:53.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:53.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:53.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:53.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:53.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:53.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:53.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:53.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:53.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:53.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:53.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:53.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:53.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:53.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:53.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:53.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:53.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:53.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:53.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:53.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:53.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:53.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:53.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:53.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:53.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:53.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:53.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:53.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:53.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:53.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:53.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:53.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:53.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:53.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:53.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:53.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:53.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:53.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:53.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:53.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:53.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:53.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:53.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:53.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:53.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:53.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:53.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:53.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:53.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:53.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:53.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:53.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:53.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:53.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:54.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:54.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:54.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:54.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:54.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:54.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:54.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:54.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:54.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:54.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:54.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:54.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:54.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:54.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:54.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:54.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:54.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:54.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:54.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:54.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:54.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:54.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:54.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:54.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:54.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:54.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:54.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:54.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:54.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:54.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:54.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:54.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:54.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:54.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:54.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:54.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:54.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:54.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:54.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:54.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:54.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:54.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:54.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:54.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:54.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:54.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:54.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:54.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:54.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:54.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:54.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:54.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:54.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:54.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:54.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:54.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:54.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:54.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:54.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:54.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:54.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:54.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:54.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:54.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:54.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:54.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:54.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:54.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=125, tag_tid=26, entry_tid=8653], [object_number=124, tag_tid=26, entry_tid=8652], [object_number=127, tag_tid=26, entry_tid=8651], [object_number=126, tag_tid=26, entry_tid=8650]]], state=connected] 2024-04-23T21:29:54.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:54.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:54.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:54.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:54.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:54.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:54.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:54.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:54.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:54.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:54.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:54.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:54.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:54.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:54.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:54.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:54.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:54.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:54.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:54.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:54.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:54.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=125, tag_tid=26, entry_tid=8653], [object_number=124, tag_tid=26, entry_tid=8652], [object_number=127, tag_tid=26, entry_tid=8651], [object_number=126, tag_tid=26, entry_tid=8650]]], state=connected] 2024-04-23T21:29:54.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:54.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=125, tag_tid=26, entry_tid=8653' 2024-04-23T21:29:54.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=125, tag_tid=26, entry_tid=8653' 2024-04-23T21:29:54.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=125, tag_tid=26, entry_tid=8653' '!=' 'object_number=126, tag_tid=26, entry_tid=8402' 2024-04-23T21:29:54.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:54.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=125, tag_tid=26, entry_tid=8653' = 'object_number=126, tag_tid=26, entry_tid=8402' 2024-04-23T21:29:54.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=125, tag_tid=26, entry_tid=8653' 2024-04-23T21:29:54.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:54.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:54.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:54.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:54.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=26, entry_tid=8653' 2024-04-23T21:29:54.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:54.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:54.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:54.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:54.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:54.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:54.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:54.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=26, entry_tid=8653' 2024-04-23T21:29:54.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:54.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:54.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=8653 2024-04-23T21:29:54.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 8653 2024-04-23T21:29:54.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:54.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:54.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:54.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:54.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:54.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:54.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:54.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:54.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:54.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:54.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:54.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:54.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:54.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:54.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:54.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:54.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:54.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:54.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:54.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:54.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:54.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:54.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:54.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:54.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:54.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:54.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:54.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:54.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:54.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:54.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:54.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:54.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:54.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:54.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:54.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:54.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:54.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:54.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:54.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:54.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:54.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:54.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:54.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:54.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:54.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:54.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:54.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:54.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:54.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:54.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:54.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:54.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:54.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:54.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:54.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:54.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:54.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:54.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:54.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:54.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:54.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:54.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:54.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:54.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:54.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:54.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:54.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:54.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:54.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:54.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:54.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:54.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:54.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:54.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:54.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:54.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:54.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:54.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:54.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:54.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:54.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:54.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:54.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:54.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:54.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:54.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:54.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:54.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:54.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:54.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:54.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:54.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:54.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:54.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:54.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:54.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:54.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:54.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:54.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:54.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:54.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:54.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:54.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:54.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:54.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:54.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:54.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:54.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:54.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:54.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:54.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:54.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:54.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:54.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:54.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:55.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:55.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:55.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:55.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:55.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=124, tag_tid=26, entry_tid=8912], [object_number=127, tag_tid=26, entry_tid=8911], [object_number=126, tag_tid=26, entry_tid=8910], [object_number=125, tag_tid=26, entry_tid=8909]]], state=connected] 2024-04-23T21:29:55.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:55.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:55.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:55.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:55.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:55.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:55.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:55.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:55.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:55.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:55.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:55.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:55.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:55.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:55.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:55.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:55.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:55.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:55.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:55.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:55.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:55.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=124, tag_tid=26, entry_tid=8912], [object_number=127, tag_tid=26, entry_tid=8911], [object_number=126, tag_tid=26, entry_tid=8910], [object_number=125, tag_tid=26, entry_tid=8909]]], state=connected] 2024-04-23T21:29:55.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:55.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=124, tag_tid=26, entry_tid=8912' 2024-04-23T21:29:55.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=124, tag_tid=26, entry_tid=8912' 2024-04-23T21:29:55.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=124, tag_tid=26, entry_tid=8912' '!=' 'object_number=125, tag_tid=26, entry_tid=8653' 2024-04-23T21:29:55.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:55.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=124, tag_tid=26, entry_tid=8912' = 'object_number=125, tag_tid=26, entry_tid=8653' 2024-04-23T21:29:55.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=124, tag_tid=26, entry_tid=8912' 2024-04-23T21:29:55.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:55.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:55.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:55.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:55.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=124, tag_tid=26, entry_tid=8912' 2024-04-23T21:29:55.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:55.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:55.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:55.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:55.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:55.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:55.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:55.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=124, tag_tid=26, entry_tid=8912' 2024-04-23T21:29:55.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:55.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:55.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=8912 2024-04-23T21:29:55.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 8912 2024-04-23T21:29:55.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:55.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:55.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:55.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:55.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:55.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:55.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:55.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:55.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:55.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:55.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:55.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:55.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:55.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:55.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:55.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:55.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:55.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:55.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:55.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:55.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:55.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:55.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:55.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:55.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:55.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:55.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:55.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:55.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:55.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:55.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:55.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:55.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:55.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:55.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:55.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:55.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:55.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:55.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:55.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:55.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:55.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:55.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:55.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:55.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:55.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:55.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:55.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:55.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:55.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:55.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:55.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:55.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:55.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:55.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:55.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:55.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:55.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:55.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:55.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:55.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:55.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:55.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:55.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:55.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:55.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:55.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:55.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:55.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:55.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:55.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:55.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:55.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:55.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:55.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:55.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:55.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:55.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:55.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:55.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:55.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:55.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:55.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:55.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:55.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:55.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:55.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:55.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:55.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:55.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:55.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:55.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:55.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:55.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:55.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:55.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:55.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:55.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:55.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:55.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:55.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:55.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:55.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:55.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:55.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:55.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:55.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:55.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:55.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:55.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:55.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:55.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:55.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:55.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:55.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:55.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:55.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:55.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:55.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:55.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:55.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:55.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:55.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:55.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:55.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:55.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:55.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:55.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:55.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:55.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:55.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:55.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:55.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:55.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:55.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:55.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:55.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:55.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=124, tag_tid=26, entry_tid=9152], [object_number=127, tag_tid=26, entry_tid=9151], [object_number=126, tag_tid=26, entry_tid=9150], [object_number=125, tag_tid=26, entry_tid=9149]]], state=connected] 2024-04-23T21:29:55.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:55.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:55.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:55.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:55.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:55.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:55.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:55.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=124, tag_tid=26, entry_tid=9152], [object_number=127, tag_tid=26, entry_tid=9151], [object_number=126, tag_tid=26, entry_tid=9150], [object_number=125, tag_tid=26, entry_tid=9149]]], state=connected] 2024-04-23T21:29:55.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:55.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=124, tag_tid=26, entry_tid=9152' 2024-04-23T21:29:55.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=124, tag_tid=26, entry_tid=9152' 2024-04-23T21:29:55.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=124, tag_tid=26, entry_tid=9152' '!=' 'object_number=124, tag_tid=26, entry_tid=8912' 2024-04-23T21:29:55.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:55.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=124, tag_tid=26, entry_tid=9152' = 'object_number=124, tag_tid=26, entry_tid=8912' 2024-04-23T21:29:55.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=124, tag_tid=26, entry_tid=9152' 2024-04-23T21:29:55.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:55.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:55.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:55.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:55.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=124, tag_tid=26, entry_tid=9152' 2024-04-23T21:29:55.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:55.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:55.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:55.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:55.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:55.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:55.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:55.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=124, tag_tid=26, entry_tid=9152' 2024-04-23T21:29:55.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:55.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=9152 2024-04-23T21:29:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 9152 2024-04-23T21:29:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:55.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:55.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:55.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:55.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:55.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:55.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:55.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:55.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:55.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:55.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:55.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:55.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:55.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:55.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:55.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:55.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:55.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:55.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:55.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:55.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:55.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:55.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:55.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:55.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:55.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:55.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:55.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:55.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:55.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:55.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:55.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:55.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:55.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:55.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:55.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:55.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:55.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:56.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:56.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:56.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:56.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:56.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:56.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:56.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:56.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:56.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:56.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:56.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:56.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:56.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:56.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:56.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:56.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:56.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:56.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:56.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:56.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:56.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:56.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:56.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:56.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:56.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:56.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:56.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:56.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:56.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:56.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:56.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:56.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:56.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:56.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:56.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:56.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:56.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:56.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:56.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:56.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:56.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:56.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:56.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:56.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:56.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:56.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:56.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:56.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:56.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:56.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:56.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:56.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:56.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:56.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:56.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:56.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:56.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:56.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:56.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:56.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:56.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:56.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:56.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:56.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:56.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:56.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:56.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:56.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:56.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:56.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:56.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:56.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:56.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:56.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:56.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:56.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:56.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:56.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:56.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:56.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:56.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:56.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:56.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:56.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:56.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:56.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:56.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:56.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:56.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:56.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:56.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:56.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:56.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:56.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:56.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:56.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:56.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:56.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:56.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:56.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:56.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:56.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:56.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:56.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:56.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:56.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:56.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:56.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:56.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:56.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:56.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:56.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:56.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:56.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:56.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:56.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:56.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:56.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:56.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:56.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:56.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=124, tag_tid=26, entry_tid=9436], [object_number=127, tag_tid=26, entry_tid=9435], [object_number=126, tag_tid=26, entry_tid=9434], [object_number=125, tag_tid=26, entry_tid=9433]]], state=connected] 2024-04-23T21:29:56.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:56.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:56.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:56.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:56.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:56.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:56.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:56.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:56.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:56.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:56.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:56.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:56.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:56.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:56.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:56.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:56.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:56.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:56.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:56.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:56.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:56.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=124, tag_tid=26, entry_tid=9436], [object_number=127, tag_tid=26, entry_tid=9435], [object_number=126, tag_tid=26, entry_tid=9434], [object_number=125, tag_tid=26, entry_tid=9433]]], state=connected] 2024-04-23T21:29:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:56.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=124, tag_tid=26, entry_tid=9436' 2024-04-23T21:29:56.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=124, tag_tid=26, entry_tid=9436' 2024-04-23T21:29:56.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=124, tag_tid=26, entry_tid=9436' '!=' 'object_number=124, tag_tid=26, entry_tid=9152' 2024-04-23T21:29:56.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:56.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=124, tag_tid=26, entry_tid=9436' = 'object_number=124, tag_tid=26, entry_tid=9152' 2024-04-23T21:29:56.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=124, tag_tid=26, entry_tid=9436' 2024-04-23T21:29:56.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:56.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:56.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:56.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:56.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=124, tag_tid=26, entry_tid=9436' 2024-04-23T21:29:56.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:56.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:56.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:56.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:56.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:56.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:56.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:56.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=124, tag_tid=26, entry_tid=9436' 2024-04-23T21:29:56.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:56.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:56.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=9436 2024-04-23T21:29:56.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 9436 2024-04-23T21:29:56.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:56.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:56.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:56.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:56.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:56.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:56.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:56.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:56.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:56.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:56.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:56.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:56.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:56.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:56.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:56.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:56.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:56.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:56.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:56.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:56.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:56.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:56.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:56.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:56.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:56.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:56.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:56.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:56.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:56.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:56.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:56.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:56.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:56.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:56.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:56.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:56.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:56.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:56.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:56.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:56.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:56.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:56.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:56.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:56.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:56.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:56.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:56.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:56.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:56.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:56.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:56.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:56.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:56.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:56.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:56.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:56.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:56.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:56.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:56.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:56.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:56.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:56.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:56.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:56.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:56.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:56.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:56.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:56.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:56.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:56.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:56.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:56.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:56.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:56.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:56.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:56.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:56.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:56.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:56.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:56.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:56.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:56.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:56.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:56.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:56.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:56.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:56.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:56.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:56.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:56.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:56.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:56.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:56.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:56.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:56.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:56.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:56.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:56.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:56.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:56.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:56.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:56.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:56.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:56.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:56.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:56.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:56.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:56.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:57.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:57.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:57.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:57.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:57.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:57.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:57.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:57.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:57.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:57.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:57.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:57.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:57.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:57.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:57.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:57.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:57.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:57.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:57.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:57.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:57.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:57.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:57.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:57.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:57.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=127, tag_tid=26, entry_tid=9715], [object_number=126, tag_tid=26, entry_tid=9714], [object_number=125, tag_tid=26, entry_tid=9713], [object_number=124, tag_tid=26, entry_tid=9712]]], state=connected] 2024-04-23T21:29:57.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:57.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:57.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:57.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:57.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:57.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:57.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:57.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:57.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:57.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:57.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:57.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:57.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:57.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:57.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:57.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:57.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:57.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:57.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:57.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:57.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:57.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=127, tag_tid=26, entry_tid=9715], [object_number=126, tag_tid=26, entry_tid=9714], [object_number=125, tag_tid=26, entry_tid=9713], [object_number=124, tag_tid=26, entry_tid=9712]]], state=connected] 2024-04-23T21:29:57.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=127, tag_tid=26, entry_tid=9715' 2024-04-23T21:29:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=127, tag_tid=26, entry_tid=9715' 2024-04-23T21:29:57.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=127, tag_tid=26, entry_tid=9715' '!=' 'object_number=124, tag_tid=26, entry_tid=9436' 2024-04-23T21:29:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=127, tag_tid=26, entry_tid=9715' = 'object_number=124, tag_tid=26, entry_tid=9436' 2024-04-23T21:29:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=127, tag_tid=26, entry_tid=9715' 2024-04-23T21:29:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:57.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:57.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=127, tag_tid=26, entry_tid=9715' 2024-04-23T21:29:57.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:57.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:57.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:57.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:57.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:57.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:57.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:57.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=127, tag_tid=26, entry_tid=9715' 2024-04-23T21:29:57.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:57.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:57.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=9715 2024-04-23T21:29:57.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 9715 2024-04-23T21:29:57.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:57.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:57.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:57.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:57.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:57.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:57.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:57.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:57.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:57.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:57.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:57.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:57.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:57.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:57.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:57.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:57.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:57.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:57.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:57.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:57.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:57.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:57.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:57.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:57.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:57.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:57.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:57.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:57.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:57.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:57.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:57.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:57.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:57.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:57.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:57.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:57.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:57.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:57.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:57.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:57.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:57.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:57.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:57.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:57.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:57.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:57.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:57.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:57.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:57.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:57.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:57.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:57.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:57.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:57.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:57.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:57.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:57.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:57.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:57.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:57.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:57.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:57.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:57.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:57.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:57.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:57.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:57.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:57.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:57.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:57.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:57.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:57.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:57.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:57.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:57.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:57.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:57.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:57.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:57.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:57.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:57.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:57.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:57.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:57.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:57.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:57.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:57.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:57.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:57.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:57.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:57.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:57.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:57.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:57.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:57.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:57.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:57.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:57.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:57.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:57.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:57.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:57.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:57.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:57.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:57.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:57.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:57.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:57.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:57.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:57.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:57.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:57.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:57.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:57.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:57.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:57.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:57.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:57.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:57.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:57.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:57.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:57.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:57.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:57.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:57.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:57.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:57.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:57.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:57.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:57.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:57.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:57.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:57.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:57.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:57.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:57.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:57.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:57.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:57.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:57.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:57.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:57.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:57.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:57.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:57.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:57.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:57.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:57.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:57.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:57.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:57.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:57.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:57.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:57.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:57.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:57.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:57.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:57.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:57.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:57.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:57.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=127, tag_tid=26, entry_tid=9979], [object_number=126, tag_tid=26, entry_tid=9978], [object_number=125, tag_tid=26, entry_tid=9977], [object_number=124, tag_tid=26, entry_tid=9976]]], state=connected] 2024-04-23T21:29:57.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:57.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:57.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:57.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:57.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:57.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:57.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:57.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:57.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:57.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:57.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:57.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:57.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:57.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:57.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:57.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:57.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:57.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:57.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:57.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:57.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:57.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=127, tag_tid=26, entry_tid=9979], [object_number=126, tag_tid=26, entry_tid=9978], [object_number=125, tag_tid=26, entry_tid=9977], [object_number=124, tag_tid=26, entry_tid=9976]]], state=connected] 2024-04-23T21:29:57.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:57.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=127, tag_tid=26, entry_tid=9979' 2024-04-23T21:29:57.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=127, tag_tid=26, entry_tid=9979' 2024-04-23T21:29:57.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=127, tag_tid=26, entry_tid=9979' '!=' 'object_number=127, tag_tid=26, entry_tid=9715' 2024-04-23T21:29:57.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:57.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=127, tag_tid=26, entry_tid=9979' = 'object_number=127, tag_tid=26, entry_tid=9715' 2024-04-23T21:29:57.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=127, tag_tid=26, entry_tid=9979' 2024-04-23T21:29:57.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:57.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:57.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:57.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:57.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=127, tag_tid=26, entry_tid=9979' 2024-04-23T21:29:57.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:57.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:57.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:57.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:57.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:57.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:57.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:57.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=127, tag_tid=26, entry_tid=9979' 2024-04-23T21:29:57.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:57.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:57.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=9979 2024-04-23T21:29:57.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 9979 2024-04-23T21:29:57.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:57.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:57.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:58.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:58.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:58.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:58.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:58.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:58.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:58.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:58.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:58.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:58.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:58.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:58.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:58.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:58.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:58.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:58.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:58.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:58.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:58.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:58.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:58.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:58.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:58.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:58.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:58.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:58.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:58.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:58.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:58.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:58.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:58.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:58.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:58.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:58.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:58.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:58.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:58.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:58.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:58.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:58.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:58.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:58.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:58.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:58.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:58.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:58.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:58.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:58.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:58.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:58.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:58.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:58.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:58.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:58.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:58.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:58.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:58.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:58.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:58.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:58.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:58.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:58.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:58.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:58.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:58.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:58.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:58.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:58.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:58.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:58.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:58.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:58.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:58.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:58.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:58.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:58.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:58.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:58.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:58.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:58.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:58.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:58.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:58.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:58.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:58.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:58.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:58.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:58.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:58.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:58.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:58.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:58.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:58.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:58.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:58.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:58.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:58.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:58.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:58.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:58.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:58.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:58.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:58.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:58.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:58.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:58.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:58.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:58.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:58.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:58.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:58.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:58.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:58.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:58.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:58.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:58.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:58.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:58.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:58.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=127, tag_tid=26, entry_tid=10251], [object_number=126, tag_tid=26, entry_tid=10250], [object_number=125, tag_tid=26, entry_tid=10249], [object_number=124, tag_tid=26, entry_tid=10248]]], state=connected] 2024-04-23T21:29:58.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:58.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:58.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:58.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:58.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:58.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:58.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:58.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:58.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:58.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:58.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:58.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:58.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:58.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:58.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:58.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:58.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:58.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:58.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:58.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:58.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:58.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=127, tag_tid=26, entry_tid=10251], [object_number=126, tag_tid=26, entry_tid=10250], [object_number=125, tag_tid=26, entry_tid=10249], [object_number=124, tag_tid=26, entry_tid=10248]]], state=connected] 2024-04-23T21:29:58.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:58.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=127, tag_tid=26, entry_tid=10251' 2024-04-23T21:29:58.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=127, tag_tid=26, entry_tid=10251' 2024-04-23T21:29:58.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=127, tag_tid=26, entry_tid=10251' '!=' 'object_number=127, tag_tid=26, entry_tid=9979' 2024-04-23T21:29:58.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:58.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=127, tag_tid=26, entry_tid=10251' = 'object_number=127, tag_tid=26, entry_tid=9979' 2024-04-23T21:29:58.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=127, tag_tid=26, entry_tid=10251' 2024-04-23T21:29:58.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:58.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:58.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:58.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:58.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=127, tag_tid=26, entry_tid=10251' 2024-04-23T21:29:58.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:58.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:58.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:58.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:58.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:58.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:58.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:58.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=127, tag_tid=26, entry_tid=10251' 2024-04-23T21:29:58.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:58.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:58.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=10251 2024-04-23T21:29:58.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 10251 2024-04-23T21:29:58.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:58.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:58.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:58.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:58.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:58.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:58.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:58.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:58.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:58.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:58.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:58.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:58.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:58.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:58.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:58.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:58.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:58.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:58.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:58.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:58.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:58.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:58.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:58.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:58.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:58.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:58.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:58.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:58.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:58.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:58.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:58.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:58.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:58.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:58.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:58.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:58.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:58.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:58.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:58.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:58.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:58.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:58.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:58.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:58.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:58.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:58.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:58.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:58.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:58.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:58.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:58.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:58.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:58.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:58.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:58.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:58.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:58.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:58.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:58.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:58.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:58.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:58.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:58.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:58.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:58.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:58.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:58.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:58.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:58.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:59.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:59.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:59.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:59.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:59.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:59.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:59.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:59.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:59.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:59.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:59.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:59.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:59.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:59.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:59.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:59.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:59.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:59.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:59.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:59.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:59.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:59.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:59.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:59.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:59.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:59.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:59.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:59.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:59.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:59.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:59.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:59.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:59.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:59.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:59.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:59.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:59.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:59.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:59.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:59.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:59.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:59.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:59.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:59.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:59.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:59.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:59.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:59.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:59.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:59.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:59.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:59.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:59.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:59.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:59.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:59.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:59.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:59.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:59.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:59.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:59.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:59.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:59.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:59.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:59.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:59.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:59.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:59.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:59.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:59.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:59.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:59.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:59.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:59.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:59.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:59.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:59.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:59.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:59.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:59.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:59.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:59.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:59.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=124, tag_tid=26, entry_tid=10448], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446], [object_number=125, tag_tid=26, entry_tid=10445]]], state=connected] 2024-04-23T21:29:59.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:59.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:59.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:59.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:59.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:59.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:59.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:59.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:59.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:59.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:59.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:59.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:59.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:59.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:59.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:59.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:59.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:59.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:59.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:59.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:59.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:59.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=124, tag_tid=26, entry_tid=10448], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446], [object_number=125, tag_tid=26, entry_tid=10445]]], state=connected] 2024-04-23T21:29:59.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:59.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=124, tag_tid=26, entry_tid=10448' 2024-04-23T21:29:59.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=124, tag_tid=26, entry_tid=10448' 2024-04-23T21:29:59.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=124, tag_tid=26, entry_tid=10448' '!=' 'object_number=127, tag_tid=26, entry_tid=10251' 2024-04-23T21:29:59.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:29:59.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=124, tag_tid=26, entry_tid=10448' = 'object_number=127, tag_tid=26, entry_tid=10251' 2024-04-23T21:29:59.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=124, tag_tid=26, entry_tid=10448' 2024-04-23T21:29:59.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:59.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:59.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:59.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=27 2024-04-23T21:29:59.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=124, tag_tid=26, entry_tid=10448' 2024-04-23T21:29:59.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:29:59.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:59.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=26 2024-04-23T21:29:59.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:59.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:59.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:59.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:29:59.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=124, tag_tid=26, entry_tid=10448' 2024-04-23T21:29:59.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:29:59.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:29:59.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=10448 2024-04-23T21:29:59.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 27 = 26 -a 1 -le 10448 2024-04-23T21:29:59.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:29:59.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:29:59.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:29:59.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:29:59.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:59.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:59.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:59.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:29:59.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:29:59.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:29:59.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:29:59.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:29:59.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:59.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:59.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:29:59.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:29:59.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:29:59.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:29:59.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:29:59.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:29:59.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:59.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:29:59.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:29:59.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:59.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:59.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:29:59.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:29:59.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:59.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:59.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:29:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:29:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:29:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:29:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:29:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:29:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:59.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:59.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:59.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:29:59.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:29:59.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:59.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:59.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:29:59.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:29:59.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:29:59.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:29:59.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:59.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:59.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:59.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:59.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:29:59.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:29:59.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:59.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:29:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:29:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:29:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:29:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:29:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:29:59.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:29:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:29:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:29:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:29:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:29:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:29:59.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:29:59.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:59.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:59.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:29:59.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:29:59.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:59.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:59.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:59.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:59.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:29:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:29:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:29:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:29:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:29:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:29:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:59.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:29:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:29:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:29:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:29:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:29:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:29:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:29:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:29:59.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:29:59.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:59.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:59.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:29:59.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:29:59.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:29:59.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:29:59.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:29:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:29:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:29:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:29:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:29:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:29:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:29:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:29:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:29:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:29:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:29:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:29:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:29:59.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:29:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:29:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:29:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:29:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:29:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:29:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:29:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:29:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:29:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:29:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:29:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:29:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:29:59.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:29:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:29:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:29:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:29:59.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:59.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:59.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:29:59.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:29:59.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:59.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:29:59.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:59.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:59.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:59.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:29:59.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:59.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:59.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:59.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:29:59.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:59.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:59.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:59.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:59.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:59.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:59.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:59.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:59.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:59.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:59.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:59.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:59.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:59.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:29:59.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:59.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:59.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:59.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:29:59.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:29:59.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:29:59.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:29:59.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:29:59.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:29:59.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:29:59.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:29:59.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:59.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:29:59.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:59.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 31 2024-04-23T21:29:59.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 31 2024-04-23T21:29:59.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:29:59.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:59.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=125, tag_tid=27, entry_tid=1], [object_number=124, tag_tid=27, entry_tid=0], [object_number=127, tag_tid=26, entry_tid=10447], [object_number=126, tag_tid=26, entry_tid=10446]]], state=connected] 2024-04-23T21:29:59.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:29:59.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:59.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=125, tag_tid=27, entry_tid=1' -a 'object_number=125, tag_tid=27, entry_tid=1' = 'object_number=125, tag_tid=27, entry_tid=1' 2024-04-23T21:29:59.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:29:59.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror test snap9 2024-04-23T21:29:59.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:29:59.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:29:59.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:29:59.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap9 2024-04-23T21:29:59.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:29:59.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:29:59.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:30:00.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/test@snap9 2024-04-23T21:30:01.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'test': 2024-04-23T21:30:01.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:30:01.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:30:01.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 17 2024-04-23T21:30:01.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10aff12cc611 2024-04-23T21:30:01.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10aff12cc611 2024-04-23T21:30:01.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:30:01.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:30:01.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:30:01.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:30:01.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:20:44 2024 2024-04-23T21:30:01.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:28:02 2024 2024-04-23T21:30:01.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:29:40 2024 2024-04-23T21:30:01.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:30:01.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10aff12cc611 2024-04-23T21:30:01.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:30:01.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:30:01.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:30:01.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:30:01.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:01.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n snap8-clean ']' 2024-04-23T21:30:01.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap8-clean 2024-04-23T21:30:01.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:01.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:01.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap8-clean 2024-04-23T21:30:01.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:30:01.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:30:01.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:01.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:01.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:30:01.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:30:01.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:01.891 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:30:01.891+0000 7fc3fc89b640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 10 2024-04-23T21:30:02.042 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:30:02.042+0000 7f91fe3f4640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 3 2024-04-23T21:30:02.114 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:30:02.115+0000 7f32a2d35640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 1 2024-04-23T21:30:02.116 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:30:02.116+0000 7fc3fa897640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 5 2024-04-23T21:30:02.238 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:30:02.239+0000 7fc3f8893640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2024-04-23T21:30:02.239 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:30:02.240+0000 7fc3f8092640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 3 2024-04-23T21:30:02.312 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:30:02.312+0000 7f32a2d35640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 2 2024-04-23T21:30:02.312 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:30:02.312+0000 7f32a3d37640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 3 2024-04-23T21:30:02.314 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:30:02.315+0000 7f32a5d3b640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 8 2024-04-23T21:30:02.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:30:02.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:02.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:02.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:02.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap9 2024-04-23T21:30:02.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:02.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:02.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap9 2024-04-23T21:30:02.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:30:02.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:30:02.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:02.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:02.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:30:03.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:30:03.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:03.505 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:30:03.504+0000 7f9204c01640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 2 2024-04-23T21:30:03.505 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:30:03.504+0000 7f91fdbf3640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 3 2024-04-23T21:30:03.505 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:30:03.505+0000 7f9200bf9640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 4 2024-04-23T21:30:03.909 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:30:03.909+0000 7fc3fc89b640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:30:03.910 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:30:03.909+0000 7fc3f8893640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2024-04-23T21:30:03.910 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:30:03.909+0000 7fc3f8092640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 3 2024-04-23T21:30:03.912 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:30:03.911+0000 7fc3f9094640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 8 2024-04-23T21:30:03.981 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:30:03.980+0000 7f32a553a640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 4 2024-04-23T21:30:04.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:30:04.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:04.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:04.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:04.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ clean_snap_name=snap9-clean 2024-04-23T21:30:04.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap9-clean 2024-04-23T21:30:04.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:30:04.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:04.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:04.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap9-clean 2024-04-23T21:30:04.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap9-clean --debug-rbd=20 --debug-journaler=20 2024-04-23T21:30:04.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:30:04.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror test 2024-04-23T21:30:04.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:30:04.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:04.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:04.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:30:04.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=9 2024-04-23T21:30:04.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:30:04.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2024-04-23T21:30:06.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 25501696 bytes 2024-04-23T21:30:08.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 47435776 bytes 2024-04-23T21:30:09.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 91843584 bytes 2024-04-23T21:30:13.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:30:13.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:30:13.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:30:13.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:13.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2024-04-23T21:30:13.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:13.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:13.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:13.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:30:13.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:30:13.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:30:13.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:30:13.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:30:14.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2024-04-23T21:30:14.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:14.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:14.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:14.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:30:14.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local description_pattern= 2024-04-23T21:30:14.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local service_pattern= 2024-04-23T21:30:14.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:30:14.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:30:14.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2024-04-23T21:30:14.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='test: 2024-04-23T21:30:14.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: global_id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:30:14.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+replaying 2024-04-23T21:30:14.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: replaying, {"bytes_per_second":3824574.53,"entries_behind_primary":1,"entries_per_second":237.57,"non_primary_position":{"entry_tid":1,"object_number":125,"tag_tid":27},"primary_position":{"entry_tid":1,"object_number":125,"tag_tid":28},"seconds_until_synced":0} 2024-04-23T21:30:14.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: service: mirror.3 on smithi150 2024-04-23T21:30:14.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:30:12 2024-04-23T21:30:14.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: peer_sites: 2024-04-23T21:30:14.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: name: cluster2 2024-04-23T21:30:14.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: state: up+stopped 2024-04-23T21:30:14.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: description: local image is primary 2024-04-23T21:30:14.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: last_update: 2024-04-23 21:30:13' 2024-04-23T21:30:14.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ state: .*up+replaying' 2024-04-23T21:30:14.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: state: up+replaying 2024-04-23T21:30:14.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^ description: .*' 2024-04-23T21:30:14.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: description: replaying, {"bytes_per_second":3824574.53,"entries_behind_primary":1,"entries_per_second":237.57,"non_primary_position":{"entry_tid":1,"object_number":125,"tag_tid":27},"primary_position":{"entry_tid":1,"object_number":125,"tag_tid":28},"seconds_until_synced":0} 2024-04-23T21:30:14.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:30:14.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:30:14.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep '^up+' 2024-04-23T21:30:14.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:30:14.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep 'service: *mirror..* on ' 2024-04-23T21:30:14.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: service: mirror.3 on smithi150 2024-04-23T21:30:14.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update 2024-04-23T21:30:14.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2024-04-23T21:30:14.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:30:12' 2024-04-23T21:30:14.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2024-04-23 21:30:12' 2024-04-23T21:30:14.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:14.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:14.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:14.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state_pattern=up+replaying 2024-04-23T21:30:14.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'prev_last_update=2024-04-23 21:30:12' 2024-04-23T21:30:14.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status 2024-04-23T21:30:14.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ CEPH_ARGS= 2024-04-23T21:30:14.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2024-04-23T21:30:15.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status='OKOKOK142224277mirror.3smithi15019.0.0-3212-g8254cb5afalseOK43434346mirror.2smithi15019.0.0-3212-g8254cb5atrueOK44594462mirror.0smithi15019.0.0-3212-g8254cb5afalseOK44954498mirror.1smithi15019.0.0-3212-g8254cb5afalseOKteste56bd005-f8a9-476d-a8fc-713b4e8cd286up+replayingreplaying, {"bytes_per_second":3824574.53,"entries_behind_primary":1,"entries_per_second":237.57,"non_primary_position":{"entry_tid":1,"object_number":125,"tag_tid":27},"primary_position":{"entry_tid":1,"object_number":125,"tag_tid":28},"seconds_until_synced":0}42224277mirror.3smithi1502024-04-23 21:30:12cluster267253cce-c621-4f62-a184-da88591999d7up+stoppedlocal image is primary2024-04-23 21:30:13' 2024-04-23T21:30:15.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_update state 2024-04-23T21:30:15.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2024-04-23T21:30:15.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_update='2024-04-23 21:30:12' 2024-04-23T21:30:15.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2024-04-23T21:30:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ state=up+replaying 2024-04-23T21:30:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo up+replaying 2024-04-23T21:30:15.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep up+replaying 2024-04-23T21:30:15.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:up+replaying 2024-04-23T21:30:15.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:15.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:15.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap10 2024-04-23T21:30:15.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap10 2024-04-23T21:30:15.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:30:15.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:15.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:15.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap10 2024-04-23T21:30:15.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap10 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:30:42.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror test 2024-04-23T21:30:42.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:42.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:42.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:42.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2024-04-23T21:30:42.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:42.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:42.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:42.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:30:42.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:30:42.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:30:42.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:30:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror test started 2024-04-23T21:30:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:30:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:30:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:30:43.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2024-04-23T21:30:43.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:30:43.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:30:43.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:43.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:30:43.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:30:43.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:30:43.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:30:43.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:30:43.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:30:43.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:30:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:30:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:30:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:30:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:30:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:30:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:30:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:43.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:30:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:30:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:30:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:30:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:30:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:30:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:30:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:30:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:30:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:30:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:30:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:30:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:30:43.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:30:43.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:30:43.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:43.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:30:43.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:30:43.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:30:43.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:30:43.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:30:43.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:43.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:43.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:43.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:30:44.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:44.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:30:44.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:44.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:44.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:30:44.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:30:44.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:30:44.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:44.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:30:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:30:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:30:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:30:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:30:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:30:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:30:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:30:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:30:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:30:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:30:44.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:30:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:30:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:30:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:30:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:30:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:30:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:30:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:30:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:30:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:30:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:30:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:30:44.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:30:44.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:44.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:30:44.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2024-04-23T21:30:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:30:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:30:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:30:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:30:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:30:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:30:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:30:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:30:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:30:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:30:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:30:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:30:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:30:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:30:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:30:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:30:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:30:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:30:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:30:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:30:44.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:44.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:30:44.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:30:44.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:30:44.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:30:44.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:30:44.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:44.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:44.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:44.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2024-04-23T21:30:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:30:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:44.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:44.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2024-04-23T21:30:44.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:30:44.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:30:44.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:44.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:30:44.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:30:44.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:30:44.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:44.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:44.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:30:44.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:44.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:30:44.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:30:44.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:30:44.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:30:44.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:30:44.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:30:44.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:30:44.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:30:44.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:30:44.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:30:44.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:30:44.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:30:44.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:30:44.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:30:44.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:44.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:30:44.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:30:44.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:30:44.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:30:44.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:30:44.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:44.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:44.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:44.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2024-04-23T21:30:44.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:30:44.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:30:44.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:30:44.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:30:44.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:30:44.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:30:44.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:30:44.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:44.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:30:44.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:30:44.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:30:44.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:44.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:44.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:30:44.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:30:44.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:30:44.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:30:44.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:44.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:44.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:30:44.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:30:44.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:30:44.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:30:44.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:30:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:30:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:30:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:30:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:30:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:30:44.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:30:44.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:44.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:44.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:30:44.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:30:44.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:30:44.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:30:44.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:30:44.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:30:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:30:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:30:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:30:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:30:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:30:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:30:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:30:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:30:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:30:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:30:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:30:44.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:44.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:44.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:44.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:30:44.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:44.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:44.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:44.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:30:44.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:30:44.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:30:44.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:44.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:44.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:44.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:30:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:30:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:30:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:30:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:30:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:44.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:44.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:30:44.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:44.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:44.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:44.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:30:44.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:30:44.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:30:44.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:30:44.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:44.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:44.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:30:44.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:30:44.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:44.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:44.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:44.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:30:44.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:44.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:44.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:44.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:30:44.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:30:44.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:30:44.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:44.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:44.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:44.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:30:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:44.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:44.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:44.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:30:44.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:30:44.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:44.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:44.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:30:44.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:30:44.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:44.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:44.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:30:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:30:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:30:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:30:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:30:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:30:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:30:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:44.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:44.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:30:44.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:44.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:44.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:44.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:30:44.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:30:44.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:30:44.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:44.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:44.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:44.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:30:44.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:44.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:44.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:44.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:30:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:30:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:30:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:30:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:30:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:44.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:30:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:30:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:30:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:30:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:30:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:30:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:30:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:44.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:44.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:30:44.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:30:44.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:30:44.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:44.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:44.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:44.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:30:44.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:44.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:30:44.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:44.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:44.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:30:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:30:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:30:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:30:44.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:30:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:30:44.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:44.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:30:44.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:30:44.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:30:44.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:44.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=134, tag_tid=29, entry_tid=9398], [object_number=133, tag_tid=29, entry_tid=9397], [object_number=132, tag_tid=29, entry_tid=9396], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:44.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:44.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:30:44.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:30:44.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:44.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:44.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:44.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:30:44.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:44.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:44.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:44.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:30:44.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:30:44.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:30:44.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:30:44.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:44.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:30:44.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:45.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:30:45.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:30:45.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:30:45.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:45.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=134, tag_tid=29, entry_tid=9398], [object_number=133, tag_tid=29, entry_tid=9397], [object_number=132, tag_tid=29, entry_tid=9396], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:45.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:45.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:45.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=133, tag_tid=30, entry_tid=1' -a 'object_number=133, tag_tid=30, entry_tid=1' = 'object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:45.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=134, tag_tid=29, entry_tid=9398' '!=' '' 2024-04-23T21:30:45.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:30:45.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=134, tag_tid=29, entry_tid=9398' = '' 2024-04-23T21:30:45.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:45.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:30:45.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:30:45.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:30:45.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=30 2024-04-23T21:30:45.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:45.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:30:45.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:30:45.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=29 2024-04-23T21:30:45.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:30:45.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:30:45.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:30:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:30:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:30:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:30:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=9398 2024-04-23T21:30:45.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 30 = 29 -a 1 -le 9398 2024-04-23T21:30:45.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:30:45.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:30:45.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:30:45.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:30:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:30:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:30:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:30:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:30:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:30:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:30:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:30:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:30:45.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:30:45.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:30:45.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:30:45.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:45.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:30:45.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:30:45.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:45.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:45.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:30:45.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:30:45.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:45.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:45.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:45.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:30:45.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:45.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:45.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:45.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:30:45.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:30:45.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:30:45.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:30:45.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:45.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:45.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:30:45.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:30:45.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:45.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:45.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:45.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:30:45.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:45.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:45.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:45.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:30:45.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:30:45.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:30:45.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:45.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:45.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:45.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:30:45.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:45.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:45.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:45.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:45.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:30:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:30:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:30:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:30:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:30:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:30:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:30:45.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:30:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:30:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:30:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:30:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:30:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:30:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:30:45.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:30:45.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:45.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:45.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:45.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:30:45.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:45.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:45.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:45.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:45.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:30:45.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:30:45.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:45.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:45.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:30:45.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:30:45.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:45.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:45.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:45.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:30:45.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:45.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:45.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:45.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:30:45.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:30:45.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:30:45.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:30:45.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:45.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:45.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:30:45.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:30:45.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:45.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:45.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:45.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:30:45.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:45.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:45.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:45.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:30:45.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:30:45.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:30:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:30:45.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:45.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:45.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:45.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:45.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:30:45.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:30:45.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:45.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:45.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:30:45.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:30:45.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:45.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:45.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:45.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:30:45.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:45.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:45.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:45.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:30:45.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:30:45.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:30:45.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:30:45.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:45.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:45.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:30:45.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:30:45.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:45.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:45.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:45.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:30:45.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:45.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:45.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:45.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:30:45.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:30:45.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:30:45.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:45.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:45.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:45.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:30:45.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:45.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:30:45.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:45.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:45.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:45.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:30:45.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:45.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:45.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:45.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:30:45.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:30:45.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:30:45.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:30:45.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:45.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:30:45.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:45.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:30:45.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:30:45.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:30:45.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:45.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=134, tag_tid=29, entry_tid=9398], [object_number=133, tag_tid=29, entry_tid=9397], [object_number=132, tag_tid=29, entry_tid=9396], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:45.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:45.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:45.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:30:45.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:30:45.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:30:45.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:30:45.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:45.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:30:45.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:45.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:30:45.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:30:45.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:30:45.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:45.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=134, tag_tid=29, entry_tid=9398], [object_number=133, tag_tid=29, entry_tid=9397], [object_number=132, tag_tid=29, entry_tid=9396], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:45.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:45.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:45.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=133, tag_tid=30, entry_tid=1' -a 'object_number=133, tag_tid=30, entry_tid=1' = 'object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:45.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=134, tag_tid=29, entry_tid=9398' '!=' 'object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:45.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:30:45.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:30:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:30:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:30:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:30:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:30:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:30:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:30:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:46.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:46.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:30:46.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:30:46.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:30:46.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:30:46.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:30:46.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:30:46.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:46.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:30:46.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:30:46.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:46.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:46.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:30:46.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:30:46.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:46.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:46.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:46.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:30:46.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:46.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:46.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:46.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:30:46.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:30:46.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:30:46.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:30:46.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:46.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:46.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:30:46.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:30:46.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:46.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:46.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:46.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:30:46.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:46.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:46.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:46.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:30:46.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:30:46.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:30:46.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:46.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:46.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:46.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:30:46.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:46.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:46.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:46.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:46.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:30:46.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:30:46.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:46.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:46.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:30:46.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:30:46.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:46.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:46.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:46.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:30:46.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:46.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:46.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:46.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:30:46.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:30:46.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:30:46.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:30:46.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:46.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:46.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:30:46.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:30:46.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:46.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:46.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:46.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:30:46.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:46.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:46.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:46.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:30:46.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:30:46.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:30:46.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:46.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:46.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:46.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:30:46.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:46.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:46.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:46.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:46.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:30:46.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:30:46.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:46.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:46.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:30:46.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:30:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:30:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:30:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:30:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:30:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:30:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:30:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:30:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:30:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:30:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:30:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:30:46.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:46.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:46.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:46.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:30:46.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:46.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:46.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:46.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:46.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:30:46.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:30:46.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:46.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:46.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:30:46.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:30:46.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:46.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:46.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:46.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:30:46.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:46.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:46.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:46.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:30:46.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:30:46.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:30:46.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:30:46.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:46.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:46.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:30:46.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:30:46.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:46.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:46.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:46.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:30:46.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:46.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:46.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:46.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:30:46.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:30:46.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:30:46.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:46.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:46.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:46.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:30:46.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:46.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:30:46.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:46.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:46.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:46.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:30:46.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:46.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:46.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:46.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:30:46.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:30:46.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:30:46.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:30:46.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:46.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:30:46.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:46.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:30:46.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:30:46.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:30:46.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:46.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=134, tag_tid=29, entry_tid=9398], [object_number=133, tag_tid=29, entry_tid=9397], [object_number=132, tag_tid=29, entry_tid=9396], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:46.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:46.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:30:46.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:30:46.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:46.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:46.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:46.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:30:46.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:46.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:46.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:46.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:30:46.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:30:46.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:30:46.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:30:46.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:46.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:30:46.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:46.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:30:46.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:30:46.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:30:46.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:46.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=134, tag_tid=29, entry_tid=9398], [object_number=133, tag_tid=29, entry_tid=9397], [object_number=132, tag_tid=29, entry_tid=9396], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:46.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:46.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:46.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=133, tag_tid=30, entry_tid=1' -a 'object_number=133, tag_tid=30, entry_tid=1' = 'object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:46.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=134, tag_tid=29, entry_tid=9398' '!=' 'object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:46.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:30:46.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:30:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:30:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:30:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:30:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:30:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:30:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:30:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:30:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:30:47.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:30:47.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:30:47.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:30:47.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:30:47.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:30:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:30:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:30:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:30:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:30:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:47.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:30:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:30:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:30:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:30:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:30:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:30:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:30:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:47.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:47.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:30:47.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:30:47.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:30:47.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:47.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:47.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:47.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:30:47.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:47.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:47.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:47.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:47.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:30:47.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:30:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:30:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:30:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:30:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:30:47.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:30:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:30:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:30:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:30:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:30:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:30:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:30:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:30:47.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:30:47.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:47.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:47.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:47.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:30:47.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:47.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:47.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:47.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:47.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:30:47.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:30:47.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:47.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:47.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:30:47.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:30:47.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:47.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:47.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:47.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:30:47.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:47.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:47.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:47.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:30:47.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:30:47.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:30:47.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:30:47.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:47.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:47.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:30:47.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:30:47.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:47.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:30:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:30:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:30:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:30:47.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:47.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:47.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:47.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:30:47.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:47.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:47.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:47.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:47.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:30:47.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:30:47.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:47.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:47.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:30:47.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:30:47.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:47.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:47.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:47.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:30:47.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:47.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:47.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:47.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:30:47.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:30:47.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:30:47.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:30:47.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:47.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:47.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:30:47.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:30:47.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:47.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:47.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:47.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:30:47.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:47.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:47.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:47.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:30:47.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:30:47.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:30:47.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:47.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:47.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:47.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:30:47.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:47.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:30:47.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:47.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:47.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:47.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:30:47.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:47.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:47.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:47.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:30:47.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:30:47.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:30:47.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:30:47.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:47.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:30:47.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:47.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:30:47.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:30:47.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:30:47.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:47.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=134, tag_tid=29, entry_tid=9398], [object_number=133, tag_tid=29, entry_tid=9397], [object_number=132, tag_tid=29, entry_tid=9396], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:47.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:47.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:30:47.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:30:47.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:47.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:47.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:47.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:30:47.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:47.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:47.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:47.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:30:47.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:30:47.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:30:47.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:30:47.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:47.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:30:47.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:47.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:30:47.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:30:47.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:30:47.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:47.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=134, tag_tid=29, entry_tid=9398], [object_number=133, tag_tid=29, entry_tid=9397], [object_number=132, tag_tid=29, entry_tid=9396], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:47.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:47.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:47.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=133, tag_tid=30, entry_tid=1' -a 'object_number=133, tag_tid=30, entry_tid=1' = 'object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:47.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=134, tag_tid=29, entry_tid=9398' '!=' 'object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:47.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:30:47.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:30:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:30:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:30:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:30:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:30:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:30:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:30:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:30:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:30:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:30:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:30:49.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:30:49.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:30:49.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:49.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:30:49.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:30:49.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:49.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:30:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:30:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:30:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:30:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:30:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:30:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:30:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:30:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:30:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:30:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:30:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:30:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:30:49.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:49.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:49.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:49.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:30:49.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:49.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:49.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:49.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:49.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:30:49.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:30:49.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:49.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:49.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:30:49.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:30:49.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:49.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:49.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:49.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:30:49.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:49.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:49.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:49.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:30:49.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:30:49.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:30:49.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:30:49.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:49.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:49.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:30:49.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:30:49.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:49.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:49.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:49.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:30:49.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:49.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:49.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:49.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:30:49.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:30:49.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:30:49.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:49.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:49.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:49.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:30:49.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:49.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:49.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:49.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:49.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:30:49.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:30:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:30:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:30:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:30:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:30:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:30:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:30:49.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:30:49.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:49.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:49.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:30:49.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:30:49.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:49.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:49.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:49.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:30:49.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:49.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:49.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:49.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:30:49.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:30:49.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:30:49.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:49.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:49.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:49.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:30:49.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:49.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:49.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:49.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:49.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:30:49.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:30:49.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:49.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:49.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:30:49.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:30:49.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:49.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:49.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:49.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:30:49.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:49.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:49.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:49.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:30:49.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:30:49.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:30:49.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:30:49.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:49.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:49.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:30:49.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:30:49.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:49.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:49.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:49.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:30:49.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:49.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:49.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:49.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:30:49.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:30:49.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:30:49.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:49.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:49.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:49.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:30:49.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:49.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:30:49.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:49.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:49.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:49.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:30:49.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:49.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:49.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:49.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:30:49.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:30:49.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:30:49.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:30:49.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:49.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:30:49.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:49.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:30:49.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:30:49.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:30:49.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:49.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=134, tag_tid=29, entry_tid=9398], [object_number=133, tag_tid=29, entry_tid=9397], [object_number=132, tag_tid=29, entry_tid=9396], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:49.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:49.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:30:49.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:30:49.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:49.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:49.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:49.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:30:49.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:49.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:49.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:49.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:30:49.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:30:49.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:30:49.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:30:49.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:49.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:30:49.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:49.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:30:49.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:30:49.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:30:49.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:49.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=134, tag_tid=29, entry_tid=9398], [object_number=133, tag_tid=29, entry_tid=9397], [object_number=132, tag_tid=29, entry_tid=9396], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:49.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=133, tag_tid=30, entry_tid=1' -a 'object_number=133, tag_tid=30, entry_tid=1' = 'object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=134, tag_tid=29, entry_tid=9398' '!=' 'object_number=134, tag_tid=29, entry_tid=9398' 2024-04-23T21:30:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:30:49.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:30:51.846 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9287625784367806) > weight (0.25) 2024-04-23T21:30:51.846 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6253898890619831) > weight (0.25) 2024-04-23T21:30:51.846 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2024-04-23T21:30:51.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2024-04-23T21:30:51.847 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2024-04-23T21:30:51.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:30:51.849+0000 7f1eb48a6640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 39370) UID: 1000 2024-04-23T21:30:51.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:30:51.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:51.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:51.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:51.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:30:51.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:30:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:30:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:30:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:30:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:30:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:30:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:30:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:30:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:30:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:30:51.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:51.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:30:51.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:30:51.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:51.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:51.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:30:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:30:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:30:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:30:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:30:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:30:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:30:51.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:30:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:30:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:30:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:30:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:30:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:30:52.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:52.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:52.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:52.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:30:52.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:52.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:52.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:52.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:52.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:30:52.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:30:52.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:52.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:52.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:30:52.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:30:52.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:52.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:52.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:52.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:30:52.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:52.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:52.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:52.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:30:52.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:30:52.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:30:52.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:30:52.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:52.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:52.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:30:52.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:30:52.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:52.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:52.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:52.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:30:52.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:52.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:52.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:52.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:30:52.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:30:52.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:30:52.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:52.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:52.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:52.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:30:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:52.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:52.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:30:52.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:30:52.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:52.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:52.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:30:52.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:30:52.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:52.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:52.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:52.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:30:52.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:52.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:52.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:52.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:30:52.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:30:52.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:30:52.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:30:52.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:52.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:52.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:30:52.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:30:52.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:52.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:52.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:52.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:30:52.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:52.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:52.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:52.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:30:52.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:30:52.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:30:52.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:52.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:52.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:52.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:30:52.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2024-04-23T21:30:52.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:52.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:30:52.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:30:52.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:52.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:52.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:30:52.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:30:52.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:52.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:52.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:52.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:30:52.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:52.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:52.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:52.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:30:52.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:30:52.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:30:52.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:30:52.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:52.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:52.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:30:52.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:30:52.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:52.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:52.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:52.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:30:52.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:52.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:52.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:52.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:30:52.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:30:52.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:30:52.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:52.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:52.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:52.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:30:52.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:52.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:30:52.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:52.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:52.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:52.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:30:52.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:52.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:52.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:52.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:30:52.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:30:52.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:30:52.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:30:52.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:52.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:30:52.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:52.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:30:52.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:30:52.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:30:52.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:52.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:52.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:52.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:30:52.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:30:52.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:52.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:52.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:52.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:30:52.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:52.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:52.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:52.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:30:52.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:30:52.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:30:52.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:30:52.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:52.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:30:52.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:30:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:30:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:30:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:52.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:30:52.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=133, tag_tid=30, entry_tid=1' -a 'object_number=133, tag_tid=30, entry_tid=1' = 'object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:30:52.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:52.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror test snap10 2024-04-23T21:30:52.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:52.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:52.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:52.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap10 2024-04-23T21:30:52.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:30:52.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:30:52.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:30:53.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/test@snap10 2024-04-23T21:30:53.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'test': 2024-04-23T21:30:53.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:30:53.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:30:53.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 19 2024-04-23T21:30:53.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10aff12cc611 2024-04-23T21:30:53.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10aff12cc611 2024-04-23T21:30:53.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:30:53.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:30:53.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:30:53.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:30:53.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:20:44 2024 2024-04-23T21:30:53.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:30:01 2024 2024-04-23T21:30:53.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:29:40 2024 2024-04-23T21:30:53.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:30:53.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10aff12cc611 2024-04-23T21:30:53.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:30:53.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:30:53.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:30:53.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:30:53.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:53.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n snap9-clean ']' 2024-04-23T21:30:53.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap9-clean 2024-04-23T21:30:53.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:53.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:53.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap9-clean 2024-04-23T21:30:53.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:30:53.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:30:53.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:53.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:53.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:30:54.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:30:54.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:54.288 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:30:54.289+0000 7fc3f8893640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 4 2024-04-23T21:30:54.642 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:30:54.643+0000 7f9201bfb640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 1 2024-04-23T21:30:54.851 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:30:54.852+0000 7f32a4538640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 2 2024-04-23T21:30:54.852 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:30:54.852+0000 7f32a8d41640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 2 2024-04-23T21:30:55.214 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:30:55.215+0000 7fc3f9094640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 3 2024-04-23T21:30:55.218 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:30:55.217+0000 7fc3f688f640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 10 2024-04-23T21:30:55.218 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:30:55.217+0000 7fc3fd09c640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 10 2024-04-23T21:30:55.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:30:55.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:55.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:55.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:55.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap10 2024-04-23T21:30:55.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:55.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:55.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap10 2024-04-23T21:30:55.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:30:55.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:30:55.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:55.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:55.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:30:56.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:30:56.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:56.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:30:56.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:57.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:30:57.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:57.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ clean_snap_name=snap10-clean 2024-04-23T21:30:57.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror test snap10-clean 2024-04-23T21:30:57.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:30:57.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:57.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:57.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap10-clean 2024-04-23T21:30:57.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap10-clean --debug-rbd=20 --debug-journaler=20 2024-04-23T21:30:57.386 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:30:57.387+0000 7f5866e13640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 10 2024-04-23T21:30:57.386 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:30:57.387+0000 7f586360c640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 8 2024-04-23T21:30:57.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror test 2024-04-23T21:30:57.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:57.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:57.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:57.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2024-04-23T21:30:57.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:57.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:57.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:57.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:30:57.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:30:57.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:30:57.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:30:58.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror test started 2024-04-23T21:30:58.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:58.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:58.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:58.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:30:58.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:30:58.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:30:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2024-04-23T21:30:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:30:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:30:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:30:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:30:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:30:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:30:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:30:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:30:58.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:30:58.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:58.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2024-04-23T21:30:58.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:30:58.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:30:58.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:58.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:30:58.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:30:58.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:30:58.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:58.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:58.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:30:58.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:58.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:30:58.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:30:58.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:30:58.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:30:58.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:30:58.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:30:58.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:30:58.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:30:58.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:30:58.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:30:58.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:30:58.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:30:58.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:30:58.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:30:58.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:58.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:30:58.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:30:58.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:30:58.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:30:58.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:30:58.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:58.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:58.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:58.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2024-04-23T21:30:59.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:59.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:30:59.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:59.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:59.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2024-04-23T21:30:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:30:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:30:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:30:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:30:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:30:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:30:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:30:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:30:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:30:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:30:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:30:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:30:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:30:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:30:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:30:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:30:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:30:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:30:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:30:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:30:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:30:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:30:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:30:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:30:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:30:59.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:59.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:59.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:59.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2024-04-23T21:30:59.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:59.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:30:59.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:59.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2024-04-23T21:30:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:30:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:30:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:30:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:30:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:30:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:30:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:30:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:30:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:30:59.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:30:59.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:30:59.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:30:59.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:30:59.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:30:59.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:30:59.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:30:59.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:30:59.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:30:59.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:30:59.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:30:59.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:59.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:30:59.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:30:59.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:30:59.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:30:59.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:30:59.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:59.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:59.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:59.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2024-04-23T21:30:59.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2024-04-23T21:30:59.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:59.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2024-04-23T21:30:59.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:30:59.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:30:59.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:59.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:30:59.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:30:59.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:30:59.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:59.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:59.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:30:59.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:59.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:30:59.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:30:59.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:30:59.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:30:59.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:30:59.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:30:59.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:30:59.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:30:59.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:30:59.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:30:59.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:30:59.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:30:59.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:30:59.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:30:59.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:59.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:30:59.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:30:59.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:30:59.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:30:59.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:30:59.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:59.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:59.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:59.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2024-04-23T21:30:59.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:30:59.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:30:59.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:30:59.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:30:59.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:30:59.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:30:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:30:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:30:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:30:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:30:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:30:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror test 2024-04-23T21:30:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:30:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:30:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:59.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:59.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:30:59.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:30:59.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:30:59.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:30:59.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:30:59.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:30:59.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:59.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:30:59.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:30:59.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:30:59.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:30:59.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:30:59.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:30:59.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:30:59.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:59.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:30:59.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:30:59.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:30:59.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:30:59.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:30:59.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:30:59.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:30:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:30:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:30:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:30:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:30:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:30:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:59.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:30:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:30:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:30:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:30:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:30:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:30:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:30:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:30:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:59.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:30:59.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:30:59.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:30:59.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:59.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:59.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:30:59.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:30:59.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:59.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:59.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:59.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:59.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:30:59.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:30:59.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:59.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:59.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:30:59.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:30:59.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:59.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:59.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:59.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:30:59.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:59.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:59.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:59.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:30:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:30:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:30:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:30:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:30:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:30:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:30:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:30:59.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:59.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:59.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:30:59.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:30:59.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:30:59.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:59.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:59.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:30:59.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:30:59.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:30:59.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:30:59.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:30:59.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:59.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:30:59.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:30:59.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:59.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:59.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:30:59.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:30:59.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:59.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:59.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:59.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:30:59.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:59.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:59.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:59.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:30:59.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:30:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:30:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:30:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:30:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:30:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:30:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:30:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:30:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:30:59.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:30:59.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:59.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:59.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:30:59.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:30:59.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2024-04-23T21:30:59.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:30:59.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:30:59.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:30:59.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:30:59.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:30:59.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:30:59.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:30:59.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:30:59.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:30:59.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:30:59.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:30:59.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:59.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:30:59.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:59.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:30:59.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:30:59.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:30:59.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:30:59.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:30:59.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:30:59.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:30:59.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:30:59.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:30:59.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:30:59.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:30:59.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:30:59.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:30:59.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:30:59.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:30:59.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:30:59.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:30:59.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:30:59.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:59.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:59.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:30:59.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:30:59.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:30:59.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:30:59.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:59.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:59.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:59.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:30:59.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:59.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:30:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:30:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:30:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:30:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:30:59.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:59.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:30:59.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:30:59.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:30:59.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=31, entry_tid=1], [object_number=132, tag_tid=31, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:59.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:30:59.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:59.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=133, tag_tid=31, entry_tid=1' 2024-04-23T21:30:59.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:30:59.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:59.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:59.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:59.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:30:59.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:30:59.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:30:59.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:30:59.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:30:59.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:30:59.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:30:59.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:30:59.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:30:59.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:30:59.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:00.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:31:00.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:31:00.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:31:00.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=31, entry_tid=1], [object_number=132, tag_tid=31, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:00.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:00.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:00.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:31:00.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=133, tag_tid=31, entry_tid=1' -a 'object_number=133, tag_tid=31, entry_tid=1' = 'object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:31:00.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=133, tag_tid=30, entry_tid=1' '!=' '' 2024-04-23T21:31:00.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:31:00.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=133, tag_tid=30, entry_tid=1' = '' 2024-04-23T21:31:00.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:31:00.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=133, tag_tid=31, entry_tid=1' 2024-04-23T21:31:00.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:31:00.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:31:00.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=31 2024-04-23T21:31:00.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:31:00.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:31:00.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:31:00.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=30 2024-04-23T21:31:00.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=133, tag_tid=31, entry_tid=1' 2024-04-23T21:31:00.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:31:00.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:31:00.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:31:00.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:31:00.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:31:00.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:31:00.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=1 2024-04-23T21:31:00.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 31 = 30 -a 1 -le 1 2024-04-23T21:31:00.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:31:00.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:31:00.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:31:00.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:31:00.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:31:00.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:00.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:00.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:31:00.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:31:00.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:31:00.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:31:00.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:31:00.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:00.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:31:00.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:31:00.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:31:00.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:31:00.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:31:00.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:31:00.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:31:00.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:31:00.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:31:00.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:31:00.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:00.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:31:00.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:31:00.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:31:00.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:31:00.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:31:00.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:31:00.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:31:00.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:31:00.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:31:00.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:00.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:31:00.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:31:00.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:31:00.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:31:00.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:00.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:31:00.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:31:00.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:31:00.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:31:00.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:31:00.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:00.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:31:00.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:31:00.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:31:00.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:31:00.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:31:00.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:31:00.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:31:00.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:31:00.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:31:00.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:31:00.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:31:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:31:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:31:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:31:00.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:31:00.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:31:00.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:31:00.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:00.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:31:00.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:31:00.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:31:00.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:31:00.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:31:00.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:31:00.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:31:00.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:31:00.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:31:00.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:00.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:31:00.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:31:00.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:31:00.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:31:00.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:00.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:31:00.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:31:00.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:31:00.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:31:00.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:31:00.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:00.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:31:00.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:31:00.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:31:00.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:31:00.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:31:00.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:31:00.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:31:00.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:31:00.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:31:00.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:31:00.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:31:00.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:31:00.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:31:00.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:31:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:31:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:31:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:31:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:31:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:31:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:31:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:31:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:31:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:31:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:31:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:31:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:31:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:31:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:31:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:31:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:31:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:31:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:31:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:31:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:31:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:31:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:31:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:31:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:31:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:31:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:31:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:31:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:31:00.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:31:00.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:31:00.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:31:00.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:31:00.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2024-04-23T21:31:00.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:31:00.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:31:00.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:31:00.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:00.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:31:00.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:31:00.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:31:00.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:31:00.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:31:00.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:31:00.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:31:00.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:31:00.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:31:00.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:00.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:31:00.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:31:00.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:31:00.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:31:00.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:00.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:31:00.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:31:00.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:31:00.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:31:00.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:31:00.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:00.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:31:00.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:31:00.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:31:00.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:31:00.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:31:00.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:31:00.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:31:00.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:31:00.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:31:00.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:31:00.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:31:00.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:31:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:31:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:31:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:31:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:31:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:31:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:31:00.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:31:00.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:00.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:31:00.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:00.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:31:00.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:31:00.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:31:00.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=31, entry_tid=1], [object_number=132, tag_tid=31, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:00.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:00.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:00.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=133, tag_tid=31, entry_tid=1' 2024-04-23T21:31:00.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:31:00.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:31:00.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:00.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:00.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:31:00.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:31:00.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:00.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:00.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:31:00.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:31:00.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:31:00.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:31:00.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:00.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:31:00.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:00.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:31:00.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:31:00.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:31:00.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=31, entry_tid=1], [object_number=132, tag_tid=31, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:00.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:00.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:00.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:31:00.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=133, tag_tid=31, entry_tid=1' -a 'object_number=133, tag_tid=31, entry_tid=1' = 'object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:31:00.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=133, tag_tid=30, entry_tid=1' '!=' 'object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:31:00.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:31:00.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:31:00.848 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2024-04-23T21:31:00.848 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:31:00.848 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2024-04-23T21:31:00.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2024-04-23T21:31:00.849 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2024-04-23T21:31:00.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2024-04-23T21:31:00.851 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2024-04-23T21:31:01.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:31:01.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:31:01.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:01.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:01.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:31:01.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:31:01.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:31:01.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:31:01.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:31:01.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:01.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:31:01.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:31:01.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:31:01.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:31:01.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:31:01.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:31:01.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:31:01.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:31:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:31:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:31:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:31:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:31:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:31:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:31:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:31:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:31:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:31:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:31:01.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:31:01.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:01.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:31:01.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:31:01.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:31:01.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:31:01.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:01.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:31:01.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:31:01.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:31:01.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:31:01.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:31:01.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:01.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:31:01.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:31:01.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:31:01.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:31:01.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:31:01.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:31:01.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:31:01.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:31:01.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:31:01.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:31:01.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:31:01.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:31:01.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:31:01.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:31:01.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:31:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:31:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:31:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:31:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:31:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:31:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:31:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:31:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:31:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:31:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:31:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:31:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:01.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:31:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:31:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:31:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:31:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:31:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:31:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:31:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:31:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:31:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:31:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:31:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:31:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:31:01.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:31:01.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:31:01.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:31:01.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:31:01.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:31:01.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:31:01.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:31:01.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:31:01.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:31:01.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:31:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:31:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:31:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:31:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:31:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:31:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:31:01.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:31:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:31:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:31:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:31:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:31:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:31:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:31:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:31:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:31:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:31:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:31:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:31:01.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:31:01.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:31:01.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:31:01.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:01.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:31:01.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:31:01.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:31:01.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:31:01.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:31:01.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:31:01.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:31:01.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:31:01.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:31:01.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:31:01.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:31:01.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:31:01.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:31:01.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:31:01.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:31:01.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:31:01.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:31:01.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:01.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:31:01.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:31:01.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:31:01.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:31:01.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:31:01.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:31:01.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:31:01.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:31:01.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:31:01.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:01.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:31:01.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:31:01.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:31:01.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:31:01.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:01.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:31:01.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:31:01.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:31:01.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:31:01.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:31:01.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:01.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:31:01.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:31:01.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:31:01.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:31:01.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:31:01.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:31:01.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:31:01.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:31:01.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:31:01.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:31:01.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:31:01.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:01.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:31:01.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:31:01.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:31:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:31:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:31:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:31:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:31:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:31:01.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:01.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:31:01.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:01.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:31:01.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:31:01.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:31:01.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=31, entry_tid=1], [object_number=132, tag_tid=31, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:01.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:01.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:01.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=133, tag_tid=31, entry_tid=1' 2024-04-23T21:31:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:31:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:31:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:31:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:31:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:31:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:31:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:31:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:31:01.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:01.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:31:01.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:01.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:31:01.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:31:01.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:31:01.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=31, entry_tid=1], [object_number=132, tag_tid=31, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:01.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:01.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:01.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:31:01.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=133, tag_tid=31, entry_tid=1' -a 'object_number=133, tag_tid=31, entry_tid=1' = 'object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:31:01.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=133, tag_tid=30, entry_tid=1' '!=' 'object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:31:01.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:31:01.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:31:02.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:31:02.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:31:02.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:02.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:02.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:31:02.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:31:02.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:31:02.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:31:02.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:31:02.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:02.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:31:02.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:31:02.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:31:02.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:31:02.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:31:02.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:31:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:31:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:31:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:31:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:31:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:31:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:31:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:31:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:31:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:31:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:31:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:31:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:31:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:31:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:31:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:31:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:31:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:31:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:31:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:31:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:31:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:31:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:31:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:31:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:31:02.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:31:02.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:31:02.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:31:02.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:31:02.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:31:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:31:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:31:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:31:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:31:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:31:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:31:02.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:31:02.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:31:02.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:31:02.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:31:02.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:02.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:31:02.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:31:02.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:31:02.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:31:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:31:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:31:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:31:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:31:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:31:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:31:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:31:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:31:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:31:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:31:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:31:02.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:31:02.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:31:02.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:31:02.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:02.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:31:02.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:31:02.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:31:02.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:31:02.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:31:02.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:31:02.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:31:02.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:31:02.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:31:02.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:31:02.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:31:02.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:31:02.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:31:02.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:31:02.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:31:02.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:31:02.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:31:02.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:02.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:31:02.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:31:02.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:31:02.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:31:02.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:31:02.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:31:02.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:31:02.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:31:02.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:31:02.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:31:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:31:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:31:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:31:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:31:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:31:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:31:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:31:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:31:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:31:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:31:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:31:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:31:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:31:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:31:02.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:31:02.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:31:02.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:31:02.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:31:02.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:31:02.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:31:02.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:31:02.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:31:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:31:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:31:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:31:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:31:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:31:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:31:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:31:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:31:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:31:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:31:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:31:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:31:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:31:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:31:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:31:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:31:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:31:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:31:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:31:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:31:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:31:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:31:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:31:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:31:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:31:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:31:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:31:02.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:31:02.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:31:02.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:31:02.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:31:02.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:31:02.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:02.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:31:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:31:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:31:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:31:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:31:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:31:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:31:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:31:02.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:02.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:31:02.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:02.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:31:02.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:31:02.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:31:02.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=31, entry_tid=1], [object_number=132, tag_tid=31, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:02.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:02.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:02.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=133, tag_tid=31, entry_tid=1' 2024-04-23T21:31:02.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:31:02.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:31:02.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:02.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:02.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:31:02.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:31:02.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:02.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:02.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:31:02.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:31:02.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:31:02.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:31:02.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:02.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:31:02.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:03.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:31:03.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:31:03.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:31:03.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=31, entry_tid=1], [object_number=132, tag_tid=31, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:03.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=133, tag_tid=30, entry_tid=1], [object_number=132, tag_tid=30, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:03.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:03.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:31:03.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=133, tag_tid=31, entry_tid=1' -a 'object_number=133, tag_tid=31, entry_tid=1' = 'object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:31:03.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=133, tag_tid=30, entry_tid=1' '!=' 'object_number=133, tag_tid=30, entry_tid=1' 2024-04-23T21:31:03.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:31:03.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:31:04.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror test 2024-04-23T21:31:04.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:31:04.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:04.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:04.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:31:04.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n test ']' 2024-04-23T21:31:04.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/test' 2024-04-23T21:31:04.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/test 2024-04-23T21:31:04.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:31:04.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:04.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:31:04.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:31:04.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:31:04.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:31:04.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:31:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:31:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:31:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:31:04.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/test 2024-04-23T21:31:04.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:31:04.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:04.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:31:04.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:31:04.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:31:04.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:31:04.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:31:04.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:31:04.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:31:04.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:31:04.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:31:04.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:04.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:31:04.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:31:04.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:31:04.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:31:04.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:04.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:31:04.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:31:04.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:31:04.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:31:04.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:31:04.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:04.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:31:04.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:31:04.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:31:04.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:31:04.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:31:04.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:31:04.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:31:04.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:31:04.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:31:04.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:31:04.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/test 2024-04-23T21:31:04.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:31:04.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:31:04.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:31:04.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:31:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/test 2024-04-23T21:31:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:31:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:31:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:31:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:31:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:31:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:31:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:31:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:31:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:31:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:31:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:31:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:31:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:31:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:31:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:31:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:31:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:31:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:31:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:31:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:31:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:31:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:31:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:31:04.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:31:04.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:31:04.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:31:04.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:31:04.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:31:04.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:31:04.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/test 2024-04-23T21:31:04.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:31:04.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:31:04.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:31:04.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:31:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/test 2024-04-23T21:31:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:31:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:31:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:31:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:31:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:31:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:31:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:31:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:31:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:31:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:31:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:04.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:31:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:31:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:31:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:31:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:31:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:31:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:31:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:31:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:31:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:31:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:31:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:31:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:31:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:31:04.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:31:04.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:31:04.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:31:04.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:31:04.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:31:04.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/test 2024-04-23T21:31:04.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/test'] 2024-04-23T21:31:04.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:31:04.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:31:04.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:31:04.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/test 2024-04-23T21:31:04.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:31:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:31:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:31:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:31:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:31:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:31:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:31:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:31:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:31:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:31:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:31:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:31:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:31:04.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:31:04.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:04.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:31:04.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:31:04.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:31:04.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:31:04.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:31:04.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:31:04.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:31:04.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:31:04.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:31:04.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:31:04.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:31:04.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:31:04.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:31:05.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:31:05.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:31:05.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:31:05.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/test 2024-04-23T21:31:05.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:05.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror test 2024-04-23T21:31:05.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:31:05.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:05.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:05.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '' 2024-04-23T21:31:05.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:31:05.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:05.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:05.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:31:05.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:31:05.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:31:05.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:31:05.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:05.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:31:05.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:05.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:31:05.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:31:05.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:31:05.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=31, entry_tid=1], [object_number=132, tag_tid=31, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:05.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=133, tag_tid=31, entry_tid=1], [object_number=132, tag_tid=31, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:05.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:05.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=133, tag_tid=31, entry_tid=1' 2024-04-23T21:31:05.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror test 2024-04-23T21:31:05.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:31:05.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:05.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:05.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror test '..*' 2024-04-23T21:31:05.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:31:05.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:05.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:05.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:31:05.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-test.status 2024-04-23T21:31:05.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-test.status 2024-04-23T21:31:05.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:31:05.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:05.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/test 2024-04-23T21:31:05.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:05.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 33 2024-04-23T21:31:05.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 33 2024-04-23T21:31:05.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:31:05.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=133, tag_tid=31, entry_tid=1], [object_number=132, tag_tid=31, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:05.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=133, tag_tid=31, entry_tid=1], [object_number=132, tag_tid=31, entry_tid=0], [object_number=134, tag_tid=29, entry_tid=9398], [object_number=135, tag_tid=29, entry_tid=9395]]], state=connected] 2024-04-23T21:31:05.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.status 2024-04-23T21:31:05.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=133, tag_tid=31, entry_tid=1' 2024-04-23T21:31:05.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=133, tag_tid=31, entry_tid=1' -a 'object_number=133, tag_tid=31, entry_tid=1' = 'object_number=133, tag_tid=31, entry_tid=1' 2024-04-23T21:31:05.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:05.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror test snap10-clean 2024-04-23T21:31:05.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:31:05.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:05.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:05.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap10-clean 2024-04-23T21:31:05.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:31:05.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:31:05.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:31:06.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/test@snap10-clean 2024-04-23T21:31:06.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'test': 2024-04-23T21:31:06.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:31:06.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:31:06.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 20 2024-04-23T21:31:06.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10aff12cc611 2024-04-23T21:31:06.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10aff12cc611 2024-04-23T21:31:06.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:31:06.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:31:06.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:31:06.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:31:06.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:20:44 2024 2024-04-23T21:31:06.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:30:01 2024 2024-04-23T21:31:06.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:29:40 2024 2024-04-23T21:31:06.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:31:06.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10aff12cc611 2024-04-23T21:31:06.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:31:06.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:31:06.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: e56bd005-f8a9-476d-a8fc-713b4e8cd286 2024-04-23T21:31:06.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:31:06.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:06.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 1 10 2024-04-23T21:31:06.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:06.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap1 2024-04-23T21:31:06.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap1 2024-04-23T21:31:06.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:06.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:06.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap1 2024-04-23T21:31:06.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:06.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:06.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:06.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:06.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:06.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:06.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:07.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:07.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:07.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap1-clean 2024-04-23T21:31:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap1-clean 2024-04-23T21:31:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap1-clean 2024-04-23T21:31:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:07.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:07.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:07.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:07.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:08.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:08.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:08.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:08.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:08.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:08.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap2 2024-04-23T21:31:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap2 2024-04-23T21:31:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap2 2024-04-23T21:31:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:08.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:09.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:09.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:09.458 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:09.459+0000 7fc3f8893640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:31:09.459 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:09.460+0000 7fc3f9895640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 6 2024-04-23T21:31:09.739 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:31:09.739+0000 7f32a4d39640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 1 2024-04-23T21:31:09.740 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:31:09.740+0000 7f32a3d37640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 1 2024-04-23T21:31:09.740 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:31:09.740+0000 7f32a8d41640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 2 2024-04-23T21:31:09.746 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3824407583931819) > weight (0.25) 2024-04-23T21:31:09.746 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9572270170878249) > weight (0.25) 2024-04-23T21:31:09.746 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2024-04-23T21:31:09.747 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2024-04-23T21:31:09.747 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2024-04-23T21:31:09.748 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi150.stderr:2024-04-23T21:31:09.748+0000 7f0f04866640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 35499) UID: 1000 2024-04-23T21:31:10.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:10.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:10.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:10.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:10.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap2-clean 2024-04-23T21:31:10.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap2-clean 2024-04-23T21:31:10.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:10.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:10.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap2-clean 2024-04-23T21:31:10.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:10.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:10.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:10.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:10.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:10.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:10.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:11.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:11.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:11.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:11.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:11.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap3 2024-04-23T21:31:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap3 2024-04-23T21:31:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap3 2024-04-23T21:31:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:12.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:12.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:12.599 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:12.600+0000 7fc3f9094640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 4 2024-04-23T21:31:12.600 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:12.601+0000 7fc3f8893640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 4 2024-04-23T21:31:12.811 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:12.812+0000 7fc3fe09e640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:31:12.969 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:12.970+0000 7fc3fd09c640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:31:12.971 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:12.971+0000 7fc3fd89d640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 3 2024-04-23T21:31:13.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:13.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:13.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:13.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:13.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap3-clean 2024-04-23T21:31:13.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap3-clean 2024-04-23T21:31:13.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:13.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:13.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap3-clean 2024-04-23T21:31:13.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:13.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:13.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:13.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:13.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:13.748 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2024-04-23T21:31:13.748 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:31:13.748 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2024-04-23T21:31:13.749 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2024-04-23T21:31:13.749 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2024-04-23T21:31:13.752 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2024-04-23T21:31:13.752 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2024-04-23T21:31:13.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:13.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:14.142 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:14.143+0000 7fc3f9895640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 7 2024-04-23T21:31:14.143 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:14.144+0000 7fc3f8893640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 9 2024-04-23T21:31:14.360 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:14.360+0000 7fc3fc89b640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:31:14.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:14.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:14.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:14.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:14.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:14.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap4 2024-04-23T21:31:14.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap4 2024-04-23T21:31:14.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:14.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:14.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap4 2024-04-23T21:31:14.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:14.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:14.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:14.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:14.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:15.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:15.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:15.916 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:31:15.916+0000 7f32a9542640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 8 2024-04-23T21:31:16.039 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:16.040+0000 7fc3f8893640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 4 2024-04-23T21:31:16.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:16.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:16.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap4-clean 2024-04-23T21:31:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap4-clean 2024-04-23T21:31:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap4-clean 2024-04-23T21:31:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:16.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:17.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:17.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:17.325 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:17.325+0000 7fc3fa897640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 7 2024-04-23T21:31:17.325 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:17.325+0000 7fc3fc89b640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 8 2024-04-23T21:31:17.636 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:17.637+0000 7fc3fd09c640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 4 2024-04-23T21:31:17.637 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:17.637+0000 7fc3f8893640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 6 2024-04-23T21:31:17.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:17.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:18.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:18.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:18.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:18.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap5 2024-04-23T21:31:18.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap5 2024-04-23T21:31:18.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap5 2024-04-23T21:31:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:18.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:18.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:18.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:18.984 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:18.985+0000 7fc3f9895640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 3 2024-04-23T21:31:19.201 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:19.201+0000 7fc3fa096640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:31:19.201 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:19.202+0000 7fc3f8893640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2024-04-23T21:31:19.202 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:19.203+0000 7fc3f688f640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 7 2024-04-23T21:31:19.442 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:19.443+0000 7fc3f7891640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 4 2024-04-23T21:31:19.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:19.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:19.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:19.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:19.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap5-clean 2024-04-23T21:31:19.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap5-clean 2024-04-23T21:31:19.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:19.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:19.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap5-clean 2024-04-23T21:31:19.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:19.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:19.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:19.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:19.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:20.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:20.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:20.584 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:31:20.584+0000 7f91fe3f4640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 6 2024-04-23T21:31:20.584 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:31:20.585+0000 7f9204c01640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 7 2024-04-23T21:31:21.088 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:21.089+0000 7fc3fa897640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:31:21.091 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:21.091+0000 7fc3f9094640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 3 2024-04-23T21:31:21.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:21.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:21.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:21.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:21.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:21.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap6 2024-04-23T21:31:21.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap6 2024-04-23T21:31:21.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:21.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:21.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap6 2024-04-23T21:31:21.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:21.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:21.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:21.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:21.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:22.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:22.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:22.296 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:22.297+0000 7fc3fc89b640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 3 2024-04-23T21:31:22.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:22.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:23.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:23.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:23.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap6-clean 2024-04-23T21:31:23.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap6-clean 2024-04-23T21:31:23.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:23.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:23.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap6-clean 2024-04-23T21:31:23.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:23.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:23.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:23.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:23.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:23.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:23.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:24.211 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:31:24.211+0000 7f32a4538640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 3 2024-04-23T21:31:24.349 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:24.349+0000 7fc3fc89b640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 8 2024-04-23T21:31:24.426 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:31:24.425+0000 7f32a3d37640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 3 2024-04-23T21:31:24.426 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:31:24.425+0000 7f32a2d35640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 5 2024-04-23T21:31:24.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:24.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:24.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:24.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:24.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:24.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap7 2024-04-23T21:31:24.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap7 2024-04-23T21:31:24.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:24.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:24.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap7 2024-04-23T21:31:24.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:24.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:24.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:24.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:24.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:25.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:25.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:25.616 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:31:25.616+0000 7f91fe3f4640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 5 2024-04-23T21:31:25.617 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:31:25.618+0000 7f9204c01640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 12 2024-04-23T21:31:25.617 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:31:25.618+0000 7f9201bfb640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 12 2024-04-23T21:31:25.682 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:25.683+0000 7fc3f9895640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 6 2024-04-23T21:31:26.101 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:31:26.101+0000 7f32a7d3f640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 2 2024-04-23T21:31:26.101 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:31:26.102+0000 7f32a8d41640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 5 2024-04-23T21:31:26.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:26.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:26.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:26.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:26.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap7-clean 2024-04-23T21:31:26.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap7-clean 2024-04-23T21:31:26.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:26.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:26.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap7-clean 2024-04-23T21:31:26.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:26.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:26.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:26.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:26.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:27.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:27.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:27.242 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:27.242+0000 7fc3f9895640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 4 2024-04-23T21:31:27.389 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:27.390+0000 7fc3f9094640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 4 2024-04-23T21:31:27.390 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:27.391+0000 7fc3fd89d640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 6 2024-04-23T21:31:27.391 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:27.392+0000 7fc3fc89b640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 10 2024-04-23T21:31:27.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:27.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:28.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:28.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:28.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:28.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap8 2024-04-23T21:31:28.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap8 2024-04-23T21:31:28.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:28.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:28.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap8 2024-04-23T21:31:28.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:28.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:28.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:28.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:28.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:28.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:28.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:28.973 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:31:28.974+0000 7f91fdbf3640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 2 2024-04-23T21:31:29.044 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:29.045+0000 7fc3fd89d640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 4 2024-04-23T21:31:29.333 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:29.334+0000 7fc3f9094640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2024-04-23T21:31:29.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:29.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:29.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:29.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:29.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap8-clean 2024-04-23T21:31:29.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap8-clean 2024-04-23T21:31:29.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:29.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:29.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap8-clean 2024-04-23T21:31:29.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:29.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:29.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:29.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:29.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:30.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:30.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:30.428 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:30.429+0000 7fc3fc89b640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:31:30.525 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:30.525+0000 7fc3fc09a640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 4 2024-04-23T21:31:30.805 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:31:30.806+0000 7f32a4538640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 1 2024-04-23T21:31:30.806 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:31:30.806+0000 7f32a8540640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 4 2024-04-23T21:31:31.077 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:31.077+0000 7fc3fa897640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2024-04-23T21:31:31.077 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:31.077+0000 7fc3f9094640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2024-04-23T21:31:31.079 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:31.080+0000 7fc3fd09c640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:31:31.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:31.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:31.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:31.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:31.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:31.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap9 2024-04-23T21:31:31.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap9 2024-04-23T21:31:31.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:31.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:31.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap9 2024-04-23T21:31:31.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:31.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:31.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:31.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:31.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:32.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:32.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:32.575 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:32.575+0000 7fc3fc89b640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:31:32.575 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:32.576+0000 7fc3f7090640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 4 2024-04-23T21:31:32.976 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:32.977+0000 7fc3fc09a640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2024-04-23T21:31:32.977 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:32.977+0000 7fc3fc89b640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2024-04-23T21:31:33.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:33.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:33.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:33.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:33.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap9-clean 2024-04-23T21:31:33.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap9-clean 2024-04-23T21:31:33.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:33.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:33.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap9-clean 2024-04-23T21:31:33.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:33.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:33.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:33.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:33.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:34.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:34.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:34.232 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:34.233+0000 7fc3fe09e640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2024-04-23T21:31:34.602 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:34.603+0000 7fc3fd09c640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 3 2024-04-23T21:31:34.602 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:34.603+0000 7fc3f9895640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 3 2024-04-23T21:31:34.680 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:31:34.681+0000 7f32a553a640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 1 2024-04-23T21:31:34.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:34.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:35.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:35.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:35.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:35.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap10 2024-04-23T21:31:35.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap10 2024-04-23T21:31:35.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:35.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:35.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap10 2024-04-23T21:31:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:35.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10 /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:35.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:35.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10 /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:35.943 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:35.944+0000 7fc3f9895640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 5 2024-04-23T21:31:36.164 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:31:36.164+0000 7f32a8540640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 1 2024-04-23T21:31:36.164 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:31:36.165+0000 7f32a4538640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 3 2024-04-23T21:31:36.300 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:36.301+0000 7fc3f8893640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:31:36.301 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:36.301+0000 7fc3f9094640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:31:36.301 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:36.301+0000 7fc3fc89b640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 3 2024-04-23T21:31:36.372 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:36.372+0000 7fc3f9094640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:31:36.377 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:31:36.378+0000 7f32a5d3b640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 4 2024-04-23T21:31:36.377 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:31:36.378+0000 7f32a7d3f640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 4 2024-04-23T21:31:36.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:36.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:36.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:36.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:36.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap10-clean 2024-04-23T21:31:36.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror test snap10-clean 2024-04-23T21:31:36.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:36.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:36.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap10-clean 2024-04-23T21:31:36.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:31:36.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:36.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:36.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:36.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10-clean /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export 2024-04-23T21:31:37.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:37.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10-clean /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:37.606 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:31:37.606+0000 7f9204c01640 -1 bdev(0x5614d01d2e00 /var/lib/ceph/osd/cluster1-2/block) aio_submit retries 3 2024-04-23T21:31:38.014 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:31:38.014+0000 7fc3fc09a640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:31:38.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2024-04-23T21:31:38.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:38.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-test.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-test.export 2024-04-23T21:31:38.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:38.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 1 10 2024-04-23T21:31:38.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:38.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap1 2024-04-23T21:31:38.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap1 2024-04-23T21:31:38.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:38.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:38.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:38.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap1 2024-04-23T21:31:38.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap1 2024-04-23T21:31:38.662 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:38.663+0000 7f586360c640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 2 2024-04-23T21:31:38.664 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:38.666+0000 7f5866e13640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 7 2024-04-23T21:31:38.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:38.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap1-clean 2024-04-23T21:31:38.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap1-clean 2024-04-23T21:31:38.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:38.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:38.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:38.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap1-clean 2024-04-23T21:31:38.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap1-clean 2024-04-23T21:31:39.084 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:39.084+0000 7f5862e0b640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 4 2024-04-23T21:31:39.086 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:39.088+0000 7f586360c640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 11 2024-04-23T21:31:39.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:40.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap2 2024-04-23T21:31:40.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap2 2024-04-23T21:31:40.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:40.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:40.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:40.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap2 2024-04-23T21:31:40.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap2 2024-04-23T21:31:40.131 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:40.132+0000 7f5862e0b640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 3 2024-04-23T21:31:40.220 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:31:40.221+0000 7fe97bfc2640 -1 reset not still connected to 0x55a5d43b3ba0 2024-04-23T21:31:41.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:41.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap2-clean 2024-04-23T21:31:41.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap2-clean 2024-04-23T21:31:41.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:41.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:41.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:41.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap2-clean 2024-04-23T21:31:41.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap2-clean 2024-04-23T21:31:41.133 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:41.134+0000 7f585f604640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 5 2024-04-23T21:31:41.135 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:31:41.136+0000 7fe96bb66640 -1 bdev(0x55a5abf68e00 /var/lib/ceph/osd/cluster2-2/block) aio_submit retries 10 2024-04-23T21:31:41.147 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:41.148+0000 7f5866e13640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 43 2024-04-23T21:31:42.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:42.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:42.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap3 2024-04-23T21:31:42.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap3 2024-04-23T21:31:42.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:42.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:42.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:42.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap3 2024-04-23T21:31:42.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap3 2024-04-23T21:31:42.146 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:42.147+0000 7f585f604640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 11 2024-04-23T21:31:42.147 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:42.147+0000 7f5862e0b640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 10 2024-04-23T21:31:43.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:43.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap3-clean 2024-04-23T21:31:43.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap3-clean 2024-04-23T21:31:43.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:43.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:43.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:43.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap3-clean 2024-04-23T21:31:43.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap3-clean 2024-04-23T21:31:43.145 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:43.146+0000 7f585f604640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 15 2024-04-23T21:31:43.147 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:43.148+0000 7f5866e13640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 25 2024-04-23T21:31:44.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:44.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:44.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap4 2024-04-23T21:31:44.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap4 2024-04-23T21:31:44.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:44.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:44.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:44.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap4 2024-04-23T21:31:44.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap4 2024-04-23T21:31:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:45.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap4-clean 2024-04-23T21:31:45.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap4-clean 2024-04-23T21:31:45.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:45.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:45.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:45.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap4-clean 2024-04-23T21:31:45.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap4-clean 2024-04-23T21:31:46.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:46.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:46.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap5 2024-04-23T21:31:46.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap5 2024-04-23T21:31:46.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:46.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:46.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:46.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap5 2024-04-23T21:31:46.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap5 2024-04-23T21:31:46.165 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:46.166+0000 7f585f604640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 4 2024-04-23T21:31:46.165 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:46.166+0000 7f5866e13640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 3 2024-04-23T21:31:47.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:47.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap5-clean 2024-04-23T21:31:47.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap5-clean 2024-04-23T21:31:47.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:47.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:47.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:47.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap5-clean 2024-04-23T21:31:47.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap5-clean 2024-04-23T21:31:47.146 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:47.146+0000 7f5866e13640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 5 2024-04-23T21:31:47.146 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:47.147+0000 7f585f604640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 7 2024-04-23T21:31:48.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:48.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:48.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap6 2024-04-23T21:31:48.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap6 2024-04-23T21:31:48.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:48.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:48.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:48.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap6 2024-04-23T21:31:48.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap6 2024-04-23T21:31:48.397 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:48.398+0000 7f5862e0b640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 1 2024-04-23T21:31:48.404 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:48.406+0000 7f586360c640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 24 2024-04-23T21:31:49.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap6-clean 2024-04-23T21:31:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap6-clean 2024-04-23T21:31:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap6-clean 2024-04-23T21:31:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap6-clean 2024-04-23T21:31:49.352 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:49.352+0000 7f5862e0b640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 1 2024-04-23T21:31:49.353 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:49.354+0000 7f586360c640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 4 2024-04-23T21:31:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:50.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:50.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap7 2024-04-23T21:31:50.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap7 2024-04-23T21:31:50.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:50.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:50.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:50.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap7 2024-04-23T21:31:50.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap7 2024-04-23T21:31:50.391 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:50.392+0000 7f5862e0b640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 23 2024-04-23T21:31:50.393 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:50.394+0000 7f585f604640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 29 2024-04-23T21:31:51.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:51.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap7-clean 2024-04-23T21:31:51.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap7-clean 2024-04-23T21:31:51.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:51.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:51.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:51.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap7-clean 2024-04-23T21:31:51.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap7-clean 2024-04-23T21:31:51.362 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:51.363+0000 7f5866e13640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 25 2024-04-23T21:31:51.365 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:51.366+0000 7f585f604640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 33 2024-04-23T21:31:52.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:52.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:52.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap8 2024-04-23T21:31:52.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap8 2024-04-23T21:31:52.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap8 2024-04-23T21:31:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap8 2024-04-23T21:31:52.387 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:52.388+0000 7f5862e0b640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 4 2024-04-23T21:31:52.392 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:52.393+0000 7f586360c640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 8 2024-04-23T21:31:53.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:53.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap8-clean 2024-04-23T21:31:53.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap8-clean 2024-04-23T21:31:53.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:53.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:53.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:53.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap8-clean 2024-04-23T21:31:53.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap8-clean 2024-04-23T21:31:53.378 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:53.376+0000 7f585f604640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 2 2024-04-23T21:31:53.378 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:53.377+0000 7f5866e13640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 2 2024-04-23T21:31:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap9 2024-04-23T21:31:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap9 2024-04-23T21:31:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap9 2024-04-23T21:31:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap9 2024-04-23T21:31:54.414 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:54.415+0000 7f5866e13640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 6 2024-04-23T21:31:54.419 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:54.420+0000 7f586360c640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 19 2024-04-23T21:31:55.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:55.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap9-clean 2024-04-23T21:31:55.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap9-clean 2024-04-23T21:31:55.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:55.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:55.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:55.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap9-clean 2024-04-23T21:31:55.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap9-clean 2024-04-23T21:31:56.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:56.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:31:56.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap10 2024-04-23T21:31:56.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap10 2024-04-23T21:31:56.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:56.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:56.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:56.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap10 2024-04-23T21:31:56.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap10 2024-04-23T21:31:57.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:57.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap10-clean 2024-04-23T21:31:57.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror test snap10-clean 2024-04-23T21:31:57.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:57.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:57.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:57.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap10-clean 2024-04-23T21:31:57.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap10-clean 2024-04-23T21:31:57.332 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:31:57.333+0000 7f5866e13640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 3 2024-04-23T21:31:58.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:31:58.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror test 2024-04-23T21:31:58.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:58.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:58.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:58.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:31:58.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:31:58.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror test 2024-04-23T21:31:58.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:58.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:58.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:58.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/test 2024-04-23T21:31:58.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/test 2024-04-23T21:31:58.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-04-23T21:31:58.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:58.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror test deleted 2024-04-23T21:31:58.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:31:58.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:58.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:58.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:31:58.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:31:58.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:31:58.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:31:58.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror test 2024-04-23T21:31:58.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:58.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:58.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:58.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/test 2024-04-23T21:31:58.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:31:58.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image test: (2) No such file or directory 2024-04-23T21:31:58.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:31:58.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:31:58.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:31:58.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror test deleted '' 2024-04-23T21:31:58.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:31:58.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:58.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=test 2024-04-23T21:31:58.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:31:58.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:31:58.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:31:58.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:31:58.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror test 2024-04-23T21:31:58.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:31:58.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:31:58.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=test 2024-04-23T21:31:58.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/test 2024-04-23T21:31:58.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:31:58.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image test: (2) No such file or directory 2024-04-23T21:31:58.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:31:58.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:31:58.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:31:58.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:31:58.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ testlog 'TEST: create many images' 2024-04-23T21:31:58.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ tee -a /tmp/tmp.8Tyc9aSFBO/rbd-mirror.test.log 2024-04-23T21:31:58.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ date '+%F %T' 2024-04-23T21:31:58.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo 2024-04-23 21:31:58 TEST: create many images 2024-04-23T21:31:58.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:2024-04-23 21:31:58 TEST: create many images 2024-04-23T21:31:58.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ snap_name=snap 2024-04-23T21:31:58.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 1 50 2024-04-23T21:31:58.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:31:58.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_1 2024-04-23T21:31:58.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_1 journal 128M 2024-04-23T21:31:58.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:58.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:58.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:58.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:58.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:31:58.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:58.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:31:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:31:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_1 128M 2024-04-23T21:31:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:31:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:31:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:31:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:31:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:31:58.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_1 2024-04-23T21:31:58.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:31:58.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:58.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:31:58.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:31:58.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:31:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:31:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_1 2024-04-23T21:31:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:31:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:31:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:31:58.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:31:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=7 2024-04-23T21:31:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:31:58.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_1 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:32:01.852 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.46560146690694704) > weight (0.25) 2024-04-23T21:32:01.852 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.533815976793067) > weight (0.25) 2024-04-23T21:32:01.852 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6379868595876871) > weight (0.25) 2024-04-23T21:32:01.853 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.358271142513893) > weight (0.25) 2024-04-23T21:32:01.853 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2024-04-23T21:32:02.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 64980992 bytes 2024-04-23T21:32:03.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 105170944 bytes 2024-04-23T21:32:05.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 141907968 bytes 2024-04-23T21:32:05.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:32:05.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:32:05.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:32:05.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:32:05.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:32:05.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_2 2024-04-23T21:32:05.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_2 journal 128M 2024-04-23T21:32:05.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:32:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:32:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:32:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_2 128M 2024-04-23T21:32:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:32:05.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:32:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:32:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:32:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:05.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_2 2024-04-23T21:32:05.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:32:05.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:05.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:32:05.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:32:05.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:32:05.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:32:05.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_2 2024-04-23T21:32:05.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:05.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:05.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:32:05.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:32:05.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=12 2024-04-23T21:32:05.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:32:05.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_2 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:32:06.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 25040896 bytes 2024-04-23T21:32:08.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 47100928 bytes 2024-04-23T21:32:09.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 80408576 bytes 2024-04-23T21:32:10.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 85097472 bytes 2024-04-23T21:32:12.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 147651584 bytes 2024-04-23T21:32:13.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:600 IOs, 159623168 bytes 2024-04-23T21:32:13.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:700 IOs, 178883584 bytes 2024-04-23T21:32:15.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:800 IOs, 215182336 bytes 2024-04-23T21:32:17.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:32:18.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:32:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:32:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:32:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:32:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_3 2024-04-23T21:32:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_3 journal 128M 2024-04-23T21:32:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:32:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:32:18.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:32:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_3 128M 2024-04-23T21:32:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:32:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:32:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:32:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:32:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:18.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_3 2024-04-23T21:32:18.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:32:18.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:18.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:32:18.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:32:18.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:32:18.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:32:18.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_3 2024-04-23T21:32:18.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:18.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:18.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:32:18.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:32:18.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=9 2024-04-23T21:32:18.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:32:18.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_3 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:32:19.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 20645888 bytes 2024-04-23T21:32:21.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 49629184 bytes 2024-04-23T21:32:23.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 79362048 bytes 2024-04-23T21:32:23.969 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:32:23.970+0000 7f83fbfb5640 -1 reset not still connected to 0x562a72eb7380 2024-04-23T21:32:24.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 105089024 bytes 2024-04-23T21:32:24.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 117888000 bytes 2024-04-23T21:32:25.301 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:32:25.302+0000 7f83e8352640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 6 2024-04-23T21:32:25.301 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:32:25.302+0000 7f83eeb5f640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 4 2024-04-23T21:32:27.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:32:27.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:32:27.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:32:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:32:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:32:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_4 2024-04-23T21:32:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_4 journal 128M 2024-04-23T21:32:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:32:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:32:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:32:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:27.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_4 128M 2024-04-23T21:32:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:32:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:32:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:32:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:32:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_4 2024-04-23T21:32:27.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:32:27.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:27.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:32:27.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:32:27.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:32:27.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:32:27.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_4 2024-04-23T21:32:27.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:27.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:27.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:32:27.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:32:27.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=11 2024-04-23T21:32:27.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:32:27.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_4 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:32:30.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 61895680 bytes 2024-04-23T21:32:34.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 122472448 bytes 2024-04-23T21:32:35.826 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:32:35.827+0000 7f9212057640 -1 reset not still connected to 0x5614d3110f70 2024-04-23T21:32:35.827 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:32:35.827+0000 7f9212057640 -1 reset not still connected to 0x5614d3111ee0 2024-04-23T21:32:38.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 183205888 bytes 2024-04-23T21:32:38.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:32:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:32:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:32:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:32:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:32:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_5 2024-04-23T21:32:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_5 journal 128M 2024-04-23T21:32:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:32:38.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:38.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:32:38.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:32:38.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:38.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_5 128M 2024-04-23T21:32:38.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:38.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:38.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:38.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:38.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:32:38.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:38.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:32:38.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:32:38.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:32:38.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:38.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_5 2024-04-23T21:32:38.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:32:38.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:38.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:32:38.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:32:38.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:32:38.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:32:38.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_5 2024-04-23T21:32:38.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:38.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:38.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:32:38.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:32:38.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=8 2024-04-23T21:32:38.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:32:38.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_5 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:32:40.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 33560576 bytes 2024-04-23T21:32:41.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 41044992 bytes 2024-04-23T21:32:42.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 69233664 bytes 2024-04-23T21:32:45.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 108956672 bytes 2024-04-23T21:32:46.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:32:46.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:32:46.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:32:46.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:32:46.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:32:46.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_6 2024-04-23T21:32:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_6 journal 128M 2024-04-23T21:32:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:32:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:32:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:32:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_6 128M 2024-04-23T21:32:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:46.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:46.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:32:46.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:46.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:32:46.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:32:46.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:32:46.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:46.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_6 2024-04-23T21:32:46.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:32:46.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:46.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:32:46.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:32:46.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:32:46.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:32:46.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_6 2024-04-23T21:32:46.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:46.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:46.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:32:46.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:32:46.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=10 2024-04-23T21:32:46.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:32:46.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_6 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:32:50.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 66393088 bytes 2024-04-23T21:32:50.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 83228672 bytes 2024-04-23T21:32:51.753 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.49208964900185015) > weight (0.25) 2024-04-23T21:32:51.753 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2024-04-23T21:32:51.754 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2024-04-23T21:32:51.754 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2024-04-23T21:32:51.755 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:32:51.756+0000 7f34f693d640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 53495) UID: 1000 2024-04-23T21:32:53.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 109561856 bytes 2024-04-23T21:32:53.755 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2024-04-23T21:32:53.792 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:32:53.792 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2024-04-23T21:32:53.792 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2024-04-23T21:32:53.793 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2024-04-23T21:32:53.795 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2024-04-23T21:32:53.795 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2024-04-23T21:32:55.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 166836224 bytes 2024-04-23T21:32:56.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:32:56.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:32:56.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:32:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:32:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:32:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_7 2024-04-23T21:32:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_7 journal 128M 2024-04-23T21:32:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:32:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:32:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:32:56.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_7 128M 2024-04-23T21:32:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:32:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:32:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:32:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:32:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:32:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_7 2024-04-23T21:32:56.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:32:56.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:56.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:32:56.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:32:56.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:32:56.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:32:56.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_7 2024-04-23T21:32:56.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:32:56.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:32:56.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:32:56.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:32:56.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=12 2024-04-23T21:32:56.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:32:56.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_7 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:32:57.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 23206912 bytes 2024-04-23T21:33:01.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 97912832 bytes 2024-04-23T21:33:04.107 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:33:04.107+0000 7f9212057640 -1 reset not still connected to 0x561505fbea90 2024-04-23T21:33:04.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 162230272 bytes 2024-04-23T21:33:06.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 197539840 bytes 2024-04-23T21:33:06.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 254667776 bytes 2024-04-23T21:33:08.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:33:08.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:33:08.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:33:08.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:33:08.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:33:08.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_8 2024-04-23T21:33:08.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_8 journal 128M 2024-04-23T21:33:08.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:08.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:08.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:08.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:08.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:33:08.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:08.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:33:08.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:33:08.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:08.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_8 128M 2024-04-23T21:33:08.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:08.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:08.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:08.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:08.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:33:08.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:08.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:33:08.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:33:08.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:33:08.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:08.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_8 2024-04-23T21:33:08.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:33:08.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:08.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:33:08.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:33:08.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:33:08.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:33:08.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_8 2024-04-23T21:33:08.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:08.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:08.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:33:08.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:33:08.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=7 2024-04-23T21:33:08.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:33:08.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_8 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:33:08.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 3094528 bytes 2024-04-23T21:33:10.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 25976832 bytes 2024-04-23T21:33:12.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 50757632 bytes 2024-04-23T21:33:14.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 106686464 bytes 2024-04-23T21:33:15.026 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:33:15.026+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a576fd40 2024-04-23T21:33:15.101 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:33:15.102+0000 7fe96eb6c640 -1 bdev(0x55a5abf68e00 /var/lib/ceph/osd/cluster2-2/block) aio_submit retries 4 2024-04-23T21:33:15.103 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:33:15.105+0000 7fe96bb66640 -1 bdev(0x55a5abf68e00 /var/lib/ceph/osd/cluster2-2/block) aio_submit retries 4 2024-04-23T21:33:15.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:33:15.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:33:15.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:33:15.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:33:15.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:33:15.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_9 2024-04-23T21:33:15.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_9 journal 128M 2024-04-23T21:33:15.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:15.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:15.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:15.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:15.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:33:15.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:15.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:33:15.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:33:15.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:15.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_9 128M 2024-04-23T21:33:15.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:15.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:15.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:15.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:15.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:33:15.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:15.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:33:15.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:33:15.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:33:15.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:15.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_9 2024-04-23T21:33:15.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:33:15.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:15.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:33:15.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:33:15.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:33:15.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:33:15.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_9 2024-04-23T21:33:15.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:15.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:15.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:33:15.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:33:15.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=14 2024-04-23T21:33:15.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:33:15.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_9 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:33:15.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 15568896 bytes 2024-04-23T21:33:18.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 33698816 bytes 2024-04-23T21:33:18.634 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:33:18.634+0000 7fc40b4f4640 -1 reset not still connected to 0x564090cdd2b0 2024-04-23T21:33:18.637 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:33:18.636+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:33:18.637 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:33:18.637+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bb9d5930 2024-04-23T21:33:18.738 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:33:18.739+0000 7fc40b4f4640 -1 reset not still connected to 0x56408b1c1a00 2024-04-23T21:33:18.738 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:33:18.739+0000 7fc40b4f4640 -1 reset not still connected to 0x56408d04f110 2024-04-23T21:33:18.739 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:33:18.739+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc3dac30 2024-04-23T21:33:18.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 40634368 bytes 2024-04-23T21:33:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 49922048 bytes 2024-04-23T21:33:21.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 107973632 bytes 2024-04-23T21:33:22.853 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8054646669676041) > weight (0.25) 2024-04-23T21:33:22.853 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2024-04-23T21:33:22.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2024-04-23T21:33:22.854 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2024-04-23T21:33:22.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:33:22.856+0000 7fba00058640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 52551) UID: 1000 2024-04-23T21:33:23.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:600 IOs, 139978752 bytes 2024-04-23T21:33:24.855 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2024-04-23T21:33:24.855 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:33:28.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:700 IOs, 218305536 bytes 2024-04-23T21:33:29.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:33:29.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:33:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:33:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:33:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:33:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_10 2024-04-23T21:33:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_10 journal 128M 2024-04-23T21:33:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:33:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:33:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:33:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_10 128M 2024-04-23T21:33:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:29.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:33:29.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:29.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:33:29.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:33:29.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:33:29.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:29.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_10 2024-04-23T21:33:29.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:33:29.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:29.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:33:29.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:33:29.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:33:29.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:33:29.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_10 2024-04-23T21:33:29.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:29.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:29.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:33:29.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:33:29.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=7 2024-04-23T21:33:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:33:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_10 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:33:30.957 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2024-04-23T21:33:30.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2024-04-23T21:33:30.957 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2024-04-23T21:33:30.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2024-04-23T21:33:30.961 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2024-04-23T21:33:31.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 22012928 bytes 2024-04-23T21:33:33.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 44646400 bytes 2024-04-23T21:33:33.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 61638656 bytes 2024-04-23T21:33:35.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:33:35.205+0000 7f147404f640 -1 rbd::mirror::image_replayer::journal::Replayer: 0x563a461bf500 get_replay_status: replay not running 2024-04-23T21:33:36.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 88715264 bytes 2024-04-23T21:33:36.737 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:33:36.737+0000 7f32b7199640 -1 reset not still connected to 0x5617ec4f41a0 2024-04-23T21:33:36.737 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:33:36.737+0000 7f32b7199640 -1 reset not still connected to 0x5617ec4f5a00 2024-04-23T21:33:36.737 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:33:36.737+0000 7f32b7199640 -1 reset not still connected to 0x561803e01c70 2024-04-23T21:33:36.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:33:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:33:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:33:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:33:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:33:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_11 2024-04-23T21:33:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_11 journal 128M 2024-04-23T21:33:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:33:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:33:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:33:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_11 128M 2024-04-23T21:33:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:33:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:33:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:33:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:33:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:36.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_11 2024-04-23T21:33:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:33:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:33:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:33:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:33:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:33:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_11 2024-04-23T21:33:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:33:36.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:33:36.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=13 2024-04-23T21:33:36.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:33:36.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_11 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:33:41.152 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:33:41.152+0000 7fc40b4f4640 -1 reset not still connected to 0x56408d04f110 2024-04-23T21:33:41.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 63527936 bytes 2024-04-23T21:33:42.342 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:33:42.343+0000 7f83e7350640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 1 2024-04-23T21:33:43.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 98565120 bytes 2024-04-23T21:33:43.703 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:33:43.702+0000 7f32b7199640 -1 reset not still connected to 0x561824a1cb60 2024-04-23T21:33:45.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 169299968 bytes 2024-04-23T21:33:47.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 208329728 bytes 2024-04-23T21:33:49.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 250010624 bytes 2024-04-23T21:33:49.342 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:33:49.342+0000 7f9212057640 -1 reset not still connected to 0x5614d36901a0 2024-04-23T21:33:49.342 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:33:49.342+0000 7f9212057640 -1 reset not still connected to 0x5614d7fb6a90 2024-04-23T21:33:49.343 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:33:49.344+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:33:49.343 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:33:49.344+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:33:49.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:33:49.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:33:49.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:33:49.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:33:49.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:33:49.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_12 2024-04-23T21:33:49.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_12 journal 128M 2024-04-23T21:33:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:33:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:33:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:33:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_12 128M 2024-04-23T21:33:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:33:49.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:49.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:33:49.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:33:49.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:33:49.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:49.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_12 2024-04-23T21:33:49.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:33:49.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:49.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:33:49.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:33:49.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:33:49.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:33:49.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_12 2024-04-23T21:33:49.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:49.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:49.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:33:49.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:33:49.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=9 2024-04-23T21:33:49.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:33:49.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_12 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:33:51.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 22148096 bytes 2024-04-23T21:33:54.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 72283136 bytes 2024-04-23T21:33:57.035 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:33:57.035+0000 7f32b7199640 -1 reset not still connected to 0x56181105c9c0 2024-04-23T21:33:57.036 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:33:57.035+0000 7f32b7199640 -1 reset not still connected to 0x561816d2b6c0 2024-04-23T21:33:57.588 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:33:57.588+0000 7f32b7199640 -1 reset not still connected to 0x561806b6dad0 2024-04-23T21:33:57.588 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:33:57.588+0000 7f32b7199640 -1 reset not still connected to 0x56181105c9c0 2024-04-23T21:33:57.588 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:33:57.588+0000 7f32b7199640 -1 reset not still connected to 0x561816d2b6c0 2024-04-23T21:33:57.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 180440064 bytes 2024-04-23T21:33:58.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:33:58.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:33:58.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:33:58.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:33:58.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:33:58.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_13 2024-04-23T21:33:58.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_13 journal 128M 2024-04-23T21:33:58.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:58.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:58.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:58.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:58.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:33:58.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:58.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:33:58.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:33:58.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:58.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_13 128M 2024-04-23T21:33:58.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:58.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:58.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:58.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:58.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:33:58.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:58.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:33:58.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:33:58.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:33:58.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:33:58.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_13 2024-04-23T21:33:59.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:33:59.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:59.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:33:59.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:33:59.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:33:59.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:33:59.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_13 2024-04-23T21:33:59.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:33:59.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:33:59.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:33:59.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:33:59.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=12 2024-04-23T21:33:59.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:33:59.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_13 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:34:01.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 46726144 bytes 2024-04-23T21:34:04.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 65235968 bytes 2024-04-23T21:34:04.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 65841152 bytes 2024-04-23T21:34:04.781 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:04.781+0000 7fc40b4f4640 -1 reset not still connected to 0x564090a0b5f0 2024-04-23T21:34:04.781 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:04.781+0000 7fc40b4f4640 -1 reset not still connected to 0x564096727450 2024-04-23T21:34:05.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 97032192 bytes 2024-04-23T21:34:09.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 154587136 bytes 2024-04-23T21:34:09.651 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:09.652+0000 7fc40b4f4640 -1 reset not still connected to 0x564085f0eb60 2024-04-23T21:34:09.652 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:09.652+0000 7fc40b4f4640 -1 reset not still connected to 0x56408b23b520 2024-04-23T21:34:09.652 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:09.652+0000 7fc40b4f4640 -1 reset not still connected to 0x5640999bed00 2024-04-23T21:34:09.654 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:09.655+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a114be10 2024-04-23T21:34:09.654 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:09.655+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4ed8f70 2024-04-23T21:34:09.655 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:09.655+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:34:09.656 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:09.656+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc182ea0 2024-04-23T21:34:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:34:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:34:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:34:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:34:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:34:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_14 2024-04-23T21:34:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_14 journal 128M 2024-04-23T21:34:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:34:11.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:34:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:34:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:34:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_14 128M 2024-04-23T21:34:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:34:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:34:11.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:11.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:34:11.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:34:11.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:34:11.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:11.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_14 2024-04-23T21:34:11.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:34:11.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:11.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:34:11.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:34:11.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:34:11.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:34:11.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_14 2024-04-23T21:34:11.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:34:11.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:34:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:34:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=10 2024-04-23T21:34:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:34:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_14 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:34:13.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 28848128 bytes 2024-04-23T21:34:13.528 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:13.528+0000 7fc40b4f4640 -1 reset not still connected to 0x5640879b6000 2024-04-23T21:34:13.530 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:13.530+0000 7fc40b4f4640 -1 reset not still connected to 0x564089fdf6c0 2024-04-23T21:34:13.530 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:13.530+0000 7fc40b4f4640 -1 reset not still connected to 0x56408b1c1a00 2024-04-23T21:34:13.530 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:13.530+0000 7fc40b4f4640 -1 reset not still connected to 0x56408d04f110 2024-04-23T21:34:17.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 117814272 bytes 2024-04-23T21:34:19.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 181801984 bytes 2024-04-23T21:34:21.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:34:21.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:34:21.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:34:21.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:34:21.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:34:21.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_15 2024-04-23T21:34:21.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_15 journal 128M 2024-04-23T21:34:21.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:34:21.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:21.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:21.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:21.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:34:21.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:21.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:34:21.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:34:21.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:21.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_15 128M 2024-04-23T21:34:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:34:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:34:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:34:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:34:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:34:21.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:21.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_15 2024-04-23T21:34:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:34:21.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:21.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:34:21.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:34:21.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:34:21.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:34:21.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_15 2024-04-23T21:34:21.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:34:21.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:21.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:34:21.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:34:21.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=14 2024-04-23T21:34:21.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:34:21.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_15 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:34:23.208 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:34:23.208+0000 7f9212057640 -1 reset not still connected to 0x5614d3110f70 2024-04-23T21:34:23.209 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:34:23.209+0000 7f9212057640 -1 reset not still connected to 0x5614d3111d40 2024-04-23T21:34:23.209 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:34:23.209+0000 7f9212057640 -1 reset not still connected to 0x5614d3111ee0 2024-04-23T21:34:23.209 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:34:23.209+0000 7f9212057640 -1 reset not still connected to 0x5614e74b8dd0 2024-04-23T21:34:23.209 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:34:23.209+0000 7f9212057640 -1 reset not still connected to 0x5614f1d24d00 2024-04-23T21:34:23.340 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:34:23.340+0000 7f9212057640 -1 reset not still connected to 0x5614e4b7a8f0 2024-04-23T21:34:23.340 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:34:23.340+0000 7f9212057640 -1 reset not still connected to 0x5614f24eb2b0 2024-04-23T21:34:23.834 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:34:23.834+0000 7f9212057640 -1 reset not still connected to 0x5614fc0b0410 2024-04-23T21:34:23.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 71639040 bytes 2024-04-23T21:34:24.862 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:34:24.862+0000 7f32b7199640 -1 reset not still connected to 0x561816d2b6c0 2024-04-23T21:34:28.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 124002304 bytes 2024-04-23T21:34:28.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:34:28.306+0000 7f382e5c2640 -1 rbd::mirror::image_replayer::journal::Replayer: 0x55cbd7c74000 get_replay_status: replay not running 2024-04-23T21:34:29.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 152151040 bytes 2024-04-23T21:34:29.278 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:34:29.278+0000 7f32b7199640 -1 reset not still connected to 0x5617ec4f5a00 2024-04-23T21:34:29.279 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:34:29.279+0000 7f32b7199640 -1 reset not still connected to 0x561803e01c70 2024-04-23T21:34:32.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 197665792 bytes 2024-04-23T21:34:35.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:34:35.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:34:35.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:34:35.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:34:35.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:34:35.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_16 2024-04-23T21:34:35.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_16 journal 128M 2024-04-23T21:34:35.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:34:35.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:35.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:35.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:35.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:34:35.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:35.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:34:35.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:34:35.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:35.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_16 128M 2024-04-23T21:34:35.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:34:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:34:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:34:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:34:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:34:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:35.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_16 2024-04-23T21:34:35.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:34:35.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:35.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:34:35.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:34:35.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:34:35.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:34:35.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_16 2024-04-23T21:34:35.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:34:35.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:35.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:34:35.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:34:35.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=8 2024-04-23T21:34:35.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:34:35.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_16 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:34:35.521 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:34:35.522+0000 7f9212057640 -1 reset not still connected to 0x5614fc0b0410 2024-04-23T21:34:36.316 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:36.315+0000 7fc40b4f4640 -1 reset not still connected to 0x5640999bed00 2024-04-23T21:34:36.319 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:36.318+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a114be10 2024-04-23T21:34:36.319 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:36.318+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:34:37.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 30132224 bytes 2024-04-23T21:34:39.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 50800640 bytes 2024-04-23T21:34:40.796 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5382267232142162) > weight (0.25) 2024-04-23T21:34:40.796 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2024-04-23T21:34:40.797 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2024-04-23T21:34:40.797 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2024-04-23T21:34:40.798 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:34:40.798+0000 7fe6b5a9f640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 59041) UID: 1000 2024-04-23T21:34:41.798 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2024-04-23T21:34:41.798 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:34:41.809 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2024-04-23T21:34:41.810 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2024-04-23T21:34:41.810 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2024-04-23T21:34:41.854 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2024-04-23T21:34:41.854 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2024-04-23T21:34:43.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:34:43.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:34:43.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:34:43.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:34:43.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:34:43.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_17 2024-04-23T21:34:43.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_17 journal 128M 2024-04-23T21:34:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:34:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:34:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:34:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:34:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_17 128M 2024-04-23T21:34:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:34:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:43.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:43.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:34:43.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:43.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:34:43.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:34:43.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:34:43.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:43.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_17 2024-04-23T21:34:43.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:34:43.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:43.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:34:43.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:34:43.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:34:43.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:34:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_17 2024-04-23T21:34:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:34:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:34:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:34:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=5 2024-04-23T21:34:43.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:34:43.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_17 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:34:44.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:34:44.042+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851cd80 finish: resending after timeout 2024-04-23T21:34:44.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 15185920 bytes 2024-04-23T21:34:46.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 39284736 bytes 2024-04-23T21:34:48.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:34:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:34:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:34:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:34:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:34:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_18 2024-04-23T21:34:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_18 journal 128M 2024-04-23T21:34:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:34:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:34:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:48.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:34:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:34:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_18 128M 2024-04-23T21:34:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:34:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:34:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:34:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:34:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:34:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:34:48.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_18 2024-04-23T21:34:48.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:34:48.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:48.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:34:48.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:34:48.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:34:48.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:34:48.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_18 2024-04-23T21:34:48.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:34:48.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:34:48.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:34:48.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:34:48.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=13 2024-04-23T21:34:48.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:34:48.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_18 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:34:48.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 10901504 bytes 2024-04-23T21:34:50.035 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:50.035+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a114be10 2024-04-23T21:34:50.036 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:50.035+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4ed8f70 2024-04-23T21:34:50.036 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:50.035+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:34:50.037 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:50.035+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc182ea0 2024-04-23T21:34:51.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:34:51.020+0000 7f382edc3640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd7dee3c0 finish: resending after timeout 2024-04-23T21:34:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 86162432 bytes 2024-04-23T21:34:56.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 138679296 bytes 2024-04-23T21:34:57.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:34:57.572+0000 7f382edc3640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851cd80 finish: resending after timeout 2024-04-23T21:34:58.096 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:58.096+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a114be10 2024-04-23T21:34:58.096 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:58.096+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4ed8f70 2024-04-23T21:34:58.096 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:58.096+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:34:58.096 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:34:58.097+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc182ea0 2024-04-23T21:34:58.992 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:34:58.993+0000 7f32b7199640 -1 reset not still connected to 0x561824a1cb60 2024-04-23T21:35:00.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 217843712 bytes 2024-04-23T21:35:00.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:35:00.518+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd7dee3c0 finish: resending after timeout 2024-04-23T21:35:01.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:35:01.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:35:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:35:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:35:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:35:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_19 2024-04-23T21:35:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_19 journal 128M 2024-04-23T21:35:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:35:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:35:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:35:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_19 128M 2024-04-23T21:35:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:35:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:35:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:35:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:35:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_19 2024-04-23T21:35:01.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:35:01.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:35:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:35:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:35:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:35:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_19 2024-04-23T21:35:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:35:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:35:01.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=5 2024-04-23T21:35:01.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:35:01.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_19 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:35:02.448 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:35:02.449+0000 7f9212057640 -1 reset not still connected to 0x5614fc0b0410 2024-04-23T21:35:05.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 50792448 bytes 2024-04-23T21:35:05.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:35:05.519+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd7dee3c0 finish: resending after timeout 2024-04-23T21:35:05.895 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:05.895+0000 7fc40b4f4640 -1 reset not still connected to 0x56408b1c1a00 2024-04-23T21:35:05.900 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:05.900+0000 7fc40b4f4640 -1 reset not still connected to 0x56408d04f110 2024-04-23T21:35:06.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:35:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:35:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:35:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:35:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:35:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_20 2024-04-23T21:35:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_20 journal 128M 2024-04-23T21:35:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:35:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:06.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:35:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:35:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_20 128M 2024-04-23T21:35:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:35:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:35:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:35:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:35:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:06.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_20 2024-04-23T21:35:06.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:35:06.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:06.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:35:06.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:35:06.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:35:06.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:35:06.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_20 2024-04-23T21:35:06.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:06.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:06.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:35:06.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:35:06.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=8 2024-04-23T21:35:06.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:35:06.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_20 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:35:07.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:35:07.943+0000 7f382edc3640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851cd80 finish: resending after timeout 2024-04-23T21:35:08.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 37670912 bytes 2024-04-23T21:35:09.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:35:09.277+0000 7f382edc3640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd8652540 finish: resending after timeout 2024-04-23T21:35:11.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 61002752 bytes 2024-04-23T21:35:11.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 101624832 bytes 2024-04-23T21:35:14.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 126202880 bytes 2024-04-23T21:35:14.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:35:14.277+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd8652540 finish: resending after timeout 2024-04-23T21:35:14.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:35:14.331+0000 7f382edc3640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd7dee3c0 finish: resending after timeout 2024-04-23T21:35:14.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:35:14.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:35:14.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:35:14.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:35:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:35:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_21 2024-04-23T21:35:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_21 journal 128M 2024-04-23T21:35:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:35:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:35:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:35:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_21 128M 2024-04-23T21:35:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:35:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:35:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:35:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:35:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:14.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_21 2024-04-23T21:35:14.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:35:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:35:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:35:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:35:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:35:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_21 2024-04-23T21:35:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:35:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:35:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=6 2024-04-23T21:35:14.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:35:14.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_21 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:35:16.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 37121024 bytes 2024-04-23T21:35:17.909 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:17.909+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:35:17.914 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:17.915+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc182ea0 2024-04-23T21:35:18.341 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:35:18.342+0000 7f83e8b53640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 1 2024-04-23T21:35:18.346 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:35:18.347+0000 7f83e7350640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 14 2024-04-23T21:35:18.594 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:18.595+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4ed8f70 2024-04-23T21:35:18.594 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:18.595+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:35:18.596 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:18.596+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc182ea0 2024-04-23T21:35:18.962 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5216849574520527) > weight (0.25) 2024-04-23T21:35:18.962 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2024-04-23T21:35:18.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2024-04-23T21:35:18.962 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2024-04-23T21:35:18.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:35:18.964+0000 7f9065bd5640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 59467) UID: 1000 2024-04-23T21:35:19.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:35:19.280+0000 7f382edc3640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd8652540 finish: resending after timeout 2024-04-23T21:35:19.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 59063296 bytes 2024-04-23T21:35:19.964 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2024-04-23T21:35:19.964 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:35:20.223 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:35:20.224+0000 7f83e8b53640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 5 2024-04-23T21:35:20.225 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:35:20.226+0000 7f83ea356640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 8 2024-04-23T21:35:20.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:35:20.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:35:20.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:35:20.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:35:20.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:35:20.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_22 2024-04-23T21:35:20.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_22 journal 128M 2024-04-23T21:35:20.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:20.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:20.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:20.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:20.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:35:20.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:20.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:35:20.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:35:20.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:20.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_22 128M 2024-04-23T21:35:20.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:20.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:20.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:20.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:20.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:35:20.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:20.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:35:20.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:35:20.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:35:20.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:20.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_22 2024-04-23T21:35:20.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:35:20.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:20.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:35:20.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:35:20.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:35:20.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:35:20.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_22 2024-04-23T21:35:20.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:20.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:20.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:35:20.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:35:20.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=10 2024-04-23T21:35:20.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:35:20.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_22 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:35:22.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:35:22.066+0000 7f382edc3640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd7dee3c0 finish: resending after timeout 2024-04-23T21:35:22.137 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:22.137+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:35:22.139 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:22.139+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc182ea0 2024-04-23T21:35:23.916 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:35:23.916+0000 7f5874269640 -1 reset not still connected to 0x559fe5d3c750 2024-04-23T21:35:23.916 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:35:23.916+0000 7f5874269640 -1 reset not still connected to 0x559fecb28270 2024-04-23T21:35:23.916 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:35:23.916+0000 7f5874269640 -1 reset not still connected to 0x559ff26fadd0 2024-04-23T21:35:24.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:35:24.280+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd8652540 finish: resending after timeout 2024-04-23T21:35:25.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 44023808 bytes 2024-04-23T21:35:25.473 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:35:25.474+0000 7f5874269640 -1 reset not still connected to 0x559fd3393e10 2024-04-23T21:35:25.474 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:35:25.474+0000 7f5874269640 -1 reset not still connected to 0x559fd4e324e0 2024-04-23T21:35:25.474 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:35:25.474+0000 7f5874269640 -1 reset not still connected to 0x559fd4e33a00 2024-04-23T21:35:25.474 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:35:25.474+0000 7f5874269640 -1 reset not still connected to 0x559fdd985930 2024-04-23T21:35:25.474 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:35:25.474+0000 7f5874269640 -1 reset not still connected to 0x559ffa4cc000 2024-04-23T21:35:25.474 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:35:25.474+0000 7f5874269640 -1 reset not still connected to 0x559ffe21cea0 2024-04-23T21:35:27.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 62878720 bytes 2024-04-23T21:35:27.769 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:35:27.770+0000 7f32b7199640 -1 reset not still connected to 0x561803e01c70 2024-04-23T21:35:28.793 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:35:28.793+0000 7f32b7199640 -1 reset not still connected to 0x56181119c680 2024-04-23T21:35:28.794 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:35:28.794+0000 7f32b7199640 -1 reset not still connected to 0x5618140d9110 2024-04-23T21:35:29.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 104368128 bytes 2024-04-23T21:35:29.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:35:29.202+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd7dee3c0 finish: resending after timeout 2024-04-23T21:35:30.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:35:30.352+0000 7feb42447640 -1 rbd::mirror::image_replayer::journal::Replayer: 0x56512ee3e380 get_replay_status: replay not running 2024-04-23T21:35:30.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:35:30.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:35:30.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:35:30.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:35:30.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:35:30.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_23 2024-04-23T21:35:30.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_23 journal 128M 2024-04-23T21:35:30.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:30.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:30.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:30.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:30.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:35:30.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:30.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:35:30.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:35:30.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:30.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_23 128M 2024-04-23T21:35:30.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:30.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:30.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:30.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:30.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:35:30.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:30.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:35:30.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:35:30.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:35:30.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:30.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_23 2024-04-23T21:35:30.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:35:30.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:30.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:35:30.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:35:30.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:35:30.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:35:30.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_23 2024-04-23T21:35:30.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:30.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:30.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:35:30.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:35:30.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=6 2024-04-23T21:35:30.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:35:30.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_23 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:35:32.167 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2024-04-23T21:35:32.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2024-04-23T21:35:32.167 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2024-04-23T21:35:32.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2024-04-23T21:35:32.170 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2024-04-23T21:35:33.346 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:35:33.345+0000 7f9212057640 -1 reset not still connected to 0x5614ddd505b0 2024-04-23T21:35:33.346 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:35:33.347+0000 7f9212057640 -1 reset not still connected to 0x5614e4b7a8f0 2024-04-23T21:35:33.346 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:35:33.347+0000 7f9212057640 -1 reset not still connected to 0x5614f24eb2b0 2024-04-23T21:35:34.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 63029248 bytes 2024-04-23T21:35:34.687 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:35:34.687+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:35:34.687 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:35:34.687+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:35:34.687 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:35:34.687+0000 7f9212057640 -1 reset not still connected to 0x561506a85930 2024-04-23T21:35:35.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:35:35.303+0000 7f382edc3640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd8652540 finish: resending after timeout 2024-04-23T21:35:36.736 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:35:36.736+0000 7f9212057640 -1 reset not still connected to 0x5614d8b548f0 2024-04-23T21:35:36.738 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:35:36.739+0000 7f9212057640 -1 reset not still connected to 0x5614e74b8dd0 2024-04-23T21:35:36.738 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:35:36.739+0000 7f9212057640 -1 reset not still connected to 0x5614f1d24d00 2024-04-23T21:35:36.739 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:35:36.739+0000 7f9212057640 -1 reset not still connected to 0x5614fc18aea0 2024-04-23T21:35:36.741 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:35:36.742+0000 7f9212057640 -1 reset not still connected to 0x5614ff5e8b60 2024-04-23T21:35:36.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:35:36.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:35:36.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:35:36.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:35:36.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:35:36.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_24 2024-04-23T21:35:36.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_24 journal 128M 2024-04-23T21:35:36.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:36.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:36.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:36.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:36.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:35:36.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:36.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:35:36.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:35:36.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:36.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_24 128M 2024-04-23T21:35:36.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:36.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:36.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:36.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:36.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:35:36.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:36.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:35:36.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:35:36.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:35:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:36.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_24 2024-04-23T21:35:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:35:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:35:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:35:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:35:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:35:36.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_24 2024-04-23T21:35:36.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:36.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:36.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:35:36.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:35:36.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=7 2024-04-23T21:35:36.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:35:36.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_24 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:35:39.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 31535104 bytes 2024-04-23T21:35:42.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:35:42.440+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851d140 finish: resending after timeout 2024-04-23T21:35:42.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:35:42.441+0000 7f382edc3640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd8652540 finish: resending after timeout 2024-04-23T21:35:42.661 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:35:42.662+0000 7f32b7199640 -1 reset not still connected to 0x5617f727b2b0 2024-04-23T21:35:42.662 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:35:42.662+0000 7f32b7199640 -1 reset not still connected to 0x5618067e4d00 2024-04-23T21:35:42.667 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:35:42.667+0000 7f32b7199640 -1 reset not still connected to 0x561806b6dad0 2024-04-23T21:35:42.667 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:35:42.667+0000 7f32b7199640 -1 reset not still connected to 0x56181105c9c0 2024-04-23T21:35:42.667 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:35:42.667+0000 7f32b7199640 -1 reset not still connected to 0x561816d2b6c0 2024-04-23T21:35:42.667 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:35:42.667+0000 7f32b7199640 -1 reset not still connected to 0x5618275f4750 2024-04-23T21:35:43.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 77065216 bytes 2024-04-23T21:35:43.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:35:43.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:35:43.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:35:43.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:35:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:35:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_25 2024-04-23T21:35:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_25 journal 128M 2024-04-23T21:35:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:35:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:35:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:35:43.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_25 128M 2024-04-23T21:35:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:35:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:35:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:35:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:35:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:43.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_25 2024-04-23T21:35:44.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:35:44.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:44.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:35:44.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:35:44.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:35:44.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:35:44.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_25 2024-04-23T21:35:44.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:44.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:44.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:35:44.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:35:44.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=12 2024-04-23T21:35:44.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:35:44.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_25 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:35:44.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 7187456 bytes 2024-04-23T21:35:46.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 25811968 bytes 2024-04-23T21:35:49.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 61917184 bytes 2024-04-23T21:35:50.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:35:50.382+0000 7f382edc3640 -1 rbd::mirror::image_replayer::journal::Replayer: 0x55cbd89d7500 get_replay_status: replay not running 2024-04-23T21:35:51.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:35:51.376+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd7def5c0 finish: resending after timeout 2024-04-23T21:35:51.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 117295104 bytes 2024-04-23T21:35:51.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:35:51.645+0000 7feb42c48640 -1 rbd::mirror::image_replayer::journal::Replayer: 0x56512ee3e700 get_replay_status: replay not running 2024-04-23T21:35:52.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 123825152 bytes 2024-04-23T21:35:53.185 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:53.185+0000 7fc40b4f4640 -1 reset not still connected to 0x564090a0b5f0 2024-04-23T21:35:53.185 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:53.185+0000 7fc40b4f4640 -1 reset not still connected to 0x564096727450 2024-04-23T21:35:53.185 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:53.185+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a524ca90 2024-04-23T21:35:53.185 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:53.185+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aad21c70 2024-04-23T21:35:53.724 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:53.724+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ad4ce340 2024-04-23T21:35:53.725 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:53.724+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b1d38270 2024-04-23T21:35:53.725 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:53.724+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:35:53.725 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:53.724+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:35:54.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:600 IOs, 191589376 bytes 2024-04-23T21:35:55.298 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:55.299+0000 7fc40b4f4640 -1 reset not still connected to 0x564090b145b0 2024-04-23T21:35:55.299 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:55.299+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b1d38270 2024-04-23T21:35:55.302 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:55.302+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:35:55.302 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:35:55.302+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:35:55.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:35:55.348+0000 7f382edc3640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851d140 finish: resending after timeout 2024-04-23T21:35:55.855 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6053137287533424) > weight (0.25) 2024-04-23T21:35:55.855 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7281035091349548) > weight (0.25) 2024-04-23T21:35:55.855 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8561935088923895) > weight (0.25) 2024-04-23T21:35:55.855 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2024-04-23T21:35:55.856 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2024-04-23T21:35:55.856 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2024-04-23T21:35:55.857 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:35:55.857+0000 7f367a458640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 51764) UID: 1000 2024-04-23T21:35:56.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:35:56.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:35:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:35:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:35:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:35:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_26 2024-04-23T21:35:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_26 journal 128M 2024-04-23T21:35:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:35:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:35:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:35:56.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:56.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_26 128M 2024-04-23T21:35:56.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:56.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:56.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:56.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:56.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:35:56.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:56.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:35:56.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:35:56.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:35:56.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:35:56.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_26 2024-04-23T21:35:56.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:35:56.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:56.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:35:56.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:35:56.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:35:56.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:35:56.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_26 2024-04-23T21:35:56.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:35:56.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:35:56.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:35:56.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:35:56.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=7 2024-04-23T21:35:56.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:35:56.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_26 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:35:58.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 36868096 bytes 2024-04-23T21:35:59.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 37491712 bytes 2024-04-23T21:36:01.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 92499968 bytes 2024-04-23T21:36:01.625 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:01.625+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:36:02.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:36:02.497+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851d140 finish: resending after timeout 2024-04-23T21:36:03.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:36:03.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:36:03.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:36:03.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:36:03.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:36:03.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_27 2024-04-23T21:36:03.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_27 journal 128M 2024-04-23T21:36:03.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:03.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:03.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:03.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:03.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:36:03.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:03.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:36:03.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:36:03.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:03.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_27 128M 2024-04-23T21:36:03.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:03.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:03.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:03.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:03.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:36:03.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:03.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:36:03.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:36:03.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:36:03.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:03.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_27 2024-04-23T21:36:03.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:36:03.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:03.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:36:03.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:36:03.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:36:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:36:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_27 2024-04-23T21:36:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:36:03.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:36:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=14 2024-04-23T21:36:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:36:03.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_27 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:36:03.857 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2024-04-23T21:36:03.857 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:36:03.857 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2024-04-23T21:36:03.858 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2024-04-23T21:36:03.858 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2024-04-23T21:36:03.861 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2024-04-23T21:36:03.861 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2024-04-23T21:36:09.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 96027648 bytes 2024-04-23T21:36:10.110 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:10.108+0000 7f32b7199640 -1 reset not still connected to 0x56181be82d00 2024-04-23T21:36:10.110 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:10.108+0000 7f32b7199640 -1 reset not still connected to 0x56181f915a00 2024-04-23T21:36:10.110 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:10.108+0000 7f32b7199640 -1 reset not still connected to 0x561827038a90 2024-04-23T21:36:14.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 171892736 bytes 2024-04-23T21:36:16.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:36:16.064+0000 7f382e5c2640 -1 rbd::mirror::image_replayer::journal::Replayer: 0x55cbd89d6a80 get_replay_status: replay not running 2024-04-23T21:36:16.238 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:16.238+0000 7fc40b4f4640 -1 reset not still connected to 0x564090b145b0 2024-04-23T21:36:16.243 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:16.243+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b1d38270 2024-04-23T21:36:16.243 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:16.243+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:36:16.243 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:16.243+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:36:17.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:36:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:36:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:36:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:36:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:36:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_28 2024-04-23T21:36:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_28 journal 128M 2024-04-23T21:36:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:36:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:17.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:36:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:36:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_28 128M 2024-04-23T21:36:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:36:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:36:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:36:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:36:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:17.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_28 2024-04-23T21:36:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:36:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:36:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:36:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:36:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:36:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_28 2024-04-23T21:36:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:17.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:36:17.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:36:17.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=11 2024-04-23T21:36:17.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:36:17.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_28 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:36:17.795 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:17.796+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4ed8f70 2024-04-23T21:36:17.795 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:17.796+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:36:17.796 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:17.797+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc182ea0 2024-04-23T21:36:19.255 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:36:19.256+0000 7fe97bfc2640 -1 reset not still connected to 0x55a5d82b95f0 2024-04-23T21:36:19.255 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:36:19.256+0000 7fe97bfc2640 -1 reset not still connected to 0x55a5e74b5e10 2024-04-23T21:36:19.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 26749952 bytes 2024-04-23T21:36:23.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 80438272 bytes 2024-04-23T21:36:24.406 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:24.407+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4610820 2024-04-23T21:36:24.407 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:24.407+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ad4ce340 2024-04-23T21:36:24.407 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:24.407+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b1d38270 2024-04-23T21:36:24.407 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:24.407+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:36:24.407 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:24.407+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:36:24.458 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:24.458+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc182ea0 2024-04-23T21:36:24.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 104217600 bytes 2024-04-23T21:36:24.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:36:24.955+0000 7f382edc3640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd777f440 finish: resending after timeout 2024-04-23T21:36:25.821 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:36:25.821+0000 7f83fbfb5640 -1 reset not still connected to 0x562a51cae680 2024-04-23T21:36:26.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 159975424 bytes 2024-04-23T21:36:26.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:36:26.140+0000 7f382edc3640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851d140 finish: resending after timeout 2024-04-23T21:36:26.471 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:26.472+0000 7f32b7199640 -1 reset not still connected to 0x5617f727b2b0 2024-04-23T21:36:26.471 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:26.472+0000 7f32b7199640 -1 reset not still connected to 0x5618067e4d00 2024-04-23T21:36:26.472 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:26.472+0000 7f32b7199640 -1 reset not still connected to 0x5618109795f0 2024-04-23T21:36:26.478 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:26.475+0000 7f32b7199640 -1 reset not still connected to 0x561816d2b6c0 2024-04-23T21:36:28.615 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:28.612+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:36:28.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:36:28.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:36:28.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:36:28.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:36:28.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:36:28.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_29 2024-04-23T21:36:28.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_29 journal 128M 2024-04-23T21:36:28.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:28.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:28.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:36:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:36:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:36:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_29 128M 2024-04-23T21:36:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:36:28.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:28.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:36:28.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:36:28.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:36:28.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:28.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_29 2024-04-23T21:36:28.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:36:28.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:28.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:36:28.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:36:28.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:36:28.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:36:28.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_29 2024-04-23T21:36:28.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:28.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:28.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:36:28.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:36:28.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=9 2024-04-23T21:36:28.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:36:28.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_29 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:36:29.148 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:29.147+0000 7fc40b4f4640 -1 reset not still connected to 0x5640846be410 2024-04-23T21:36:29.148 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:29.148+0000 7fc40b4f4640 -1 reset not still connected to 0x564085f0eb60 2024-04-23T21:36:29.150 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:29.150+0000 7fc40b4f4640 -1 reset not still connected to 0x56408b23b520 2024-04-23T21:36:29.150 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:29.150+0000 7fc40b4f4640 -1 reset not still connected to 0x5640999bed00 2024-04-23T21:36:29.153 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:29.153+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a114be10 2024-04-23T21:36:29.153 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:29.153+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4ed8f70 2024-04-23T21:36:29.154 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:29.153+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:36:29.154 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:29.153+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bbf7b5f0 2024-04-23T21:36:29.154 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:29.153+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc182ea0 2024-04-23T21:36:30.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 20568064 bytes 2024-04-23T21:36:32.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 48307200 bytes 2024-04-23T21:36:32.902 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:32.902+0000 7fc40b4f4640 -1 reset not still connected to 0x564089fde1a0 2024-04-23T21:36:32.902 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:32.902+0000 7fc40b4f4640 -1 reset not still connected to 0x564089fdfc70 2024-04-23T21:36:32.904 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:32.904+0000 7fc40b4f4640 -1 reset not still connected to 0x564090a0b5f0 2024-04-23T21:36:32.905 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:32.906+0000 7fc40b4f4640 -1 reset not still connected to 0x564096727450 2024-04-23T21:36:32.905 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:32.906+0000 7fc40b4f4640 -1 reset not still connected to 0x56409e1b3c70 2024-04-23T21:36:32.905 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:32.906+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a524ca90 2024-04-23T21:36:32.905 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:32.906+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aad21c70 2024-04-23T21:36:33.256 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:33.255+0000 7fc40b4f4640 -1 reset not still connected to 0x564089fde1a0 2024-04-23T21:36:33.256 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:33.255+0000 7fc40b4f4640 -1 reset not still connected to 0x564089fdfc70 2024-04-23T21:36:33.258 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:33.257+0000 7fc40b4f4640 -1 reset not still connected to 0x564090a0b5f0 2024-04-23T21:36:33.258 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:33.257+0000 7fc40b4f4640 -1 reset not still connected to 0x564096727450 2024-04-23T21:36:33.258 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:33.257+0000 7fc40b4f4640 -1 reset not still connected to 0x56409e1b3c70 2024-04-23T21:36:33.258 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:33.257+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a524ca90 2024-04-23T21:36:33.259 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:33.257+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aad21c70 2024-04-23T21:36:33.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:36:33.516+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851d140 finish: resending after timeout 2024-04-23T21:36:34.121 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:34.122+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:36:34.121 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:34.122+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:36:36.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 117753856 bytes 2024-04-23T21:36:37.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:36:37.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:36:37.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:36:37.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:36:37.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:36:37.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_30 2024-04-23T21:36:37.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_30 journal 128M 2024-04-23T21:36:37.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:37.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:37.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:37.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:36:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:36:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:36:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_30 128M 2024-04-23T21:36:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:36:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:36:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:36:37.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:36:37.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:37.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_30 2024-04-23T21:36:37.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:36:37.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:37.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:36:37.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:36:37.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:36:37.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:36:37.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_30 2024-04-23T21:36:37.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:37.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:37.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:36:37.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:36:37.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=9 2024-04-23T21:36:37.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:36:37.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_30 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:36:38.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:36:38.518+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851d140 finish: resending after timeout 2024-04-23T21:36:40.094 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:36:40.092+0000 7f9212057640 -1 reset not still connected to 0x5614dddbf2b0 2024-04-23T21:36:40.094 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:36:40.092+0000 7f9212057640 -1 reset not still connected to 0x5614dddbf930 2024-04-23T21:36:40.094 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:36:40.092+0000 7f9212057640 -1 reset not still connected to 0x5614de5295f0 2024-04-23T21:36:40.094 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:36:40.092+0000 7f9212057640 -1 reset not still connected to 0x5614e35d1a00 2024-04-23T21:36:40.094 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:36:40.092+0000 7f9212057640 -1 reset not still connected to 0x5614ee548410 2024-04-23T21:36:40.095 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:36:40.095+0000 7f9212057640 -1 reset not still connected to 0x5614fc7ae410 2024-04-23T21:36:42.586 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:42.586+0000 7fc40b4f4640 -1 reset not still connected to 0x564085f0eb60 2024-04-23T21:36:42.587 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:42.586+0000 7fc40b4f4640 -1 reset not still connected to 0x5640999bed00 2024-04-23T21:36:42.591 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:42.591+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a114be10 2024-04-23T21:36:42.591 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:42.591+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4ed8f70 2024-04-23T21:36:42.591 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:42.591+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:36:42.592 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:42.591+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bbf7b5f0 2024-04-23T21:36:42.592 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:42.592+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc182ea0 2024-04-23T21:36:43.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 62295040 bytes 2024-04-23T21:36:45.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 121221120 bytes 2024-04-23T21:36:46.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:36:46.183+0000 7f382edc3640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851d140 finish: resending after timeout 2024-04-23T21:36:46.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 129475584 bytes 2024-04-23T21:36:46.800 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:46.799+0000 7fc40b4f4640 -1 reset not still connected to 0x5640999bed00 2024-04-23T21:36:46.804 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:46.801+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a114be10 2024-04-23T21:36:46.804 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:46.804+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4ed8f70 2024-04-23T21:36:46.804 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:46.804+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:36:46.804 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:46.804+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bbf7b5f0 2024-04-23T21:36:46.805 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:46.804+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc182ea0 2024-04-23T21:36:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:36:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:36:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:36:46.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:36:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:36:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_31 2024-04-23T21:36:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_31 journal 128M 2024-04-23T21:36:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:36:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:36:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:36:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_31 128M 2024-04-23T21:36:46.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:46.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:46.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:46.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:46.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:36:46.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:46.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:36:46.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:36:46.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:36:46.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:46.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_31 2024-04-23T21:36:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:36:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:36:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:36:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:36:46.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:36:46.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_31 2024-04-23T21:36:46.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:46.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:46.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:36:46.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:36:46.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=10 2024-04-23T21:36:46.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:36:46.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_31 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:36:47.296 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:36:47.297+0000 7f9212057640 -1 reset not still connected to 0x5614d9bd9790 2024-04-23T21:36:47.299 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:36:47.297+0000 7f9212057640 -1 reset not still connected to 0x5614ddd505b0 2024-04-23T21:36:47.299 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:36:47.297+0000 7f9212057640 -1 reset not still connected to 0x5614de560000 2024-04-23T21:36:47.299 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:36:47.297+0000 7f9212057640 -1 reset not still connected to 0x5614e4b7a8f0 2024-04-23T21:36:47.299 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:36:47.297+0000 7f9212057640 -1 reset not still connected to 0x5614ebce6dd0 2024-04-23T21:36:47.299 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:36:47.297+0000 7f9212057640 -1 reset not still connected to 0x5614f24eb2b0 2024-04-23T21:36:47.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 6388736 bytes 2024-04-23T21:36:49.248 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:36:49.248+0000 7f5861608640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 2 2024-04-23T21:36:49.249 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:36:49.250+0000 7f5866e13640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 4 2024-04-23T21:36:49.989 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:49.987+0000 7f32b7199640 -1 reset not still connected to 0x5617ee64dad0 2024-04-23T21:36:49.990 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:49.987+0000 7f32b7199640 -1 reset not still connected to 0x5618279e84e0 2024-04-23T21:36:51.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 49227776 bytes 2024-04-23T21:36:52.056 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:52.056+0000 7f32b7199640 -1 reset not still connected to 0x56180a12ba00 2024-04-23T21:36:52.056 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:52.056+0000 7f32b7199640 -1 reset not still connected to 0x56181119c680 2024-04-23T21:36:52.057 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:52.057+0000 7f32b7199640 -1 reset not still connected to 0x5618140d9110 2024-04-23T21:36:52.616 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:36:52.617+0000 7f585fe05640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 2 2024-04-23T21:36:53.213 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:53.213+0000 7f32b7199640 -1 reset not still connected to 0x5617f6d73040 2024-04-23T21:36:53.213 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:53.213+0000 7f32b7199640 -1 reset not still connected to 0x5617ff7956c0 2024-04-23T21:36:53.213 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:53.213+0000 7f32b7199640 -1 reset not still connected to 0x56180a12ba00 2024-04-23T21:36:53.213 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:53.213+0000 7f32b7199640 -1 reset not still connected to 0x56181119c680 2024-04-23T21:36:53.214 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:36:53.213+0000 7f32b7199640 -1 reset not still connected to 0x5618140d9110 2024-04-23T21:36:53.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 102275072 bytes 2024-04-23T21:36:56.296 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:56.295+0000 7fc40b4f4640 -1 reset not still connected to 0x5640999bed00 2024-04-23T21:36:56.296 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:56.295+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4ed8f70 2024-04-23T21:36:56.297 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:56.295+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:36:56.298 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:56.299+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc182ea0 2024-04-23T21:36:56.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 163211264 bytes 2024-04-23T21:36:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:36:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:36:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:36:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:36:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:36:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_32 2024-04-23T21:36:56.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_32 journal 128M 2024-04-23T21:36:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:36:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:36:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:36:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_32 128M 2024-04-23T21:36:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:56.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:56.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:36:56.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:56.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:36:56.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:36:56.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:36:56.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:36:56.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_32 2024-04-23T21:36:57.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:36:57.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:57.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:36:57.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:36:57.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:36:57.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:36:57.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_32 2024-04-23T21:36:57.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:36:57.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:36:57.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:36:57.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:36:57.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=12 2024-04-23T21:36:57.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:36:57.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_32 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:36:58.734 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:36:58.735+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac2d4dd0 2024-04-23T21:37:01.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 56157184 bytes 2024-04-23T21:37:02.720 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:02.720+0000 7fc40b4f4640 -1 reset not still connected to 0x564090a0b5f0 2024-04-23T21:37:02.721 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:02.720+0000 7fc40b4f4640 -1 reset not still connected to 0x564096727450 2024-04-23T21:37:02.721 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:02.720+0000 7fc40b4f4640 -1 reset not still connected to 0x56409e1b3c70 2024-04-23T21:37:02.721 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:02.720+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a524ca90 2024-04-23T21:37:02.721 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:02.720+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aad21c70 2024-04-23T21:37:03.503 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:03.503+0000 7f32b7199640 -1 reset not still connected to 0x56181105c9c0 2024-04-23T21:37:03.503 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:03.503+0000 7f32b7199640 -1 reset not still connected to 0x561816d2b6c0 2024-04-23T21:37:03.503 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:03.503+0000 7f32b7199640 -1 reset not still connected to 0x5618283af380 2024-04-23T21:37:04.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 154071040 bytes 2024-04-23T21:37:04.727 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:37:04.728+0000 7f83eab57640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 5 2024-04-23T21:37:05.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 156459008 bytes 2024-04-23T21:37:05.171 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7313006193425299) > weight (0.25) 2024-04-23T21:37:05.171 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.48025541613967404) > weight (0.25) 2024-04-23T21:37:05.171 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8129990057404909) > weight (0.25) 2024-04-23T21:37:05.172 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7610825219572173) > weight (0.25) 2024-04-23T21:37:05.172 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2024-04-23T21:37:05.935 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:05.935+0000 7f9212057640 -1 reset not still connected to 0x5614dddbf2b0 2024-04-23T21:37:05.938 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:05.939+0000 7f9212057640 -1 reset not still connected to 0x5614de5295f0 2024-04-23T21:37:05.939 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:05.939+0000 7f9212057640 -1 reset not still connected to 0x5614e072a000 2024-04-23T21:37:05.939 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:05.939+0000 7f9212057640 -1 reset not still connected to 0x5614e35d1a00 2024-04-23T21:37:05.939 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:05.939+0000 7f9212057640 -1 reset not still connected to 0x5614ee548410 2024-04-23T21:37:05.940 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:05.939+0000 7f9212057640 -1 reset not still connected to 0x5614fc7ae410 2024-04-23T21:37:06.081 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:06.081+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b1d38270 2024-04-23T21:37:06.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 180403200 bytes 2024-04-23T21:37:06.605 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:06.605+0000 7f32b7199640 -1 reset not still connected to 0x561806b6dad0 2024-04-23T21:37:06.605 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:06.605+0000 7f32b7199640 -1 reset not still connected to 0x5618109795f0 2024-04-23T21:37:06.605 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:06.605+0000 7f32b7199640 -1 reset not still connected to 0x56181105c9c0 2024-04-23T21:37:06.605 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:06.605+0000 7f32b7199640 -1 reset not still connected to 0x561816d2b6c0 2024-04-23T21:37:06.605 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:06.605+0000 7f32b7199640 -1 reset not still connected to 0x5618275f4750 2024-04-23T21:37:06.605 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:06.605+0000 7f32b7199640 -1 reset not still connected to 0x5618283af380 2024-04-23T21:37:07.221 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:07.222+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:37:08.335 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:08.336+0000 7f9212057640 -1 reset not still connected to 0x5614e35d1a00 2024-04-23T21:37:08.336 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:08.336+0000 7f9212057640 -1 reset not still connected to 0x5614ee548410 2024-04-23T21:37:08.339 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:08.339+0000 7f9212057640 -1 reset not still connected to 0x5614fc7ae410 2024-04-23T21:37:09.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:37:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:37:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:37:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:37:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:37:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_33 2024-04-23T21:37:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_33 journal 128M 2024-04-23T21:37:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:37:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:37:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:37:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:37:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_33 128M 2024-04-23T21:37:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:37:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:37:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:37:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:37:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:37:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_33 2024-04-23T21:37:09.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:37:09.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:09.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:37:09.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:37:09.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:37:09.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:37:09.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_33 2024-04-23T21:37:09.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:37:09.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:09.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:37:09.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:37:09.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=11 2024-04-23T21:37:09.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:37:09.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_33 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:37:13.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 58827776 bytes 2024-04-23T21:37:13.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 59445248 bytes 2024-04-23T21:37:14.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 83815424 bytes 2024-04-23T21:37:17.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 132953088 bytes 2024-04-23T21:37:18.421 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:18.421+0000 7f32b7199640 -1 reset not still connected to 0x5617f727b2b0 2024-04-23T21:37:18.421 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:18.421+0000 7f32b7199640 -1 reset not still connected to 0x5618067e4d00 2024-04-23T21:37:18.426 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:18.426+0000 7f32b7199640 -1 reset not still connected to 0x561806b6dad0 2024-04-23T21:37:18.426 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:18.426+0000 7f32b7199640 -1 reset not still connected to 0x5618109795f0 2024-04-23T21:37:18.426 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:18.426+0000 7f32b7199640 -1 reset not still connected to 0x56181105c9c0 2024-04-23T21:37:18.426 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:18.426+0000 7f32b7199640 -1 reset not still connected to 0x561816d2b6c0 2024-04-23T21:37:18.426 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:18.426+0000 7f32b7199640 -1 reset not still connected to 0x5618283af380 2024-04-23T21:37:19.437 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:19.437+0000 7f9212057640 -1 reset not still connected to 0x5614f36add40 2024-04-23T21:37:19.440 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:19.439+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:37:19.441 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:19.439+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:37:19.442 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:19.442+0000 7f9212057640 -1 reset not still connected to 0x561506553790 2024-04-23T21:37:19.442 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:19.442+0000 7f9212057640 -1 reset not still connected to 0x561506a85930 2024-04-23T21:37:19.862 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2024-04-23T21:37:19.863 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2024-04-23T21:37:19.864 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2024-04-23T21:37:19.865 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:37:19.864+0000 7f7c425ce640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 49674) UID: 1000 2024-04-23T21:37:20.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:37:20.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:37:20.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:37:20.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:37:20.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:37:20.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_34 2024-04-23T21:37:20.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_34 journal 128M 2024-04-23T21:37:20.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:37:20.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:20.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:20.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:20.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:37:20.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:20.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:37:20.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:37:20.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:20.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_34 128M 2024-04-23T21:37:20.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:37:20.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:20.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:20.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:20.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:37:20.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:20.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:37:20.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:37:20.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:37:20.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:20.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_34 2024-04-23T21:37:20.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:37:20.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:20.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:37:20.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:37:20.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:37:20.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:37:20.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_34 2024-04-23T21:37:20.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:37:20.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:20.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:37:20.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:37:20.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=13 2024-04-23T21:37:20.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:37:20.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_34 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:37:21.268 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:21.267+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:37:21.268 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:21.267+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bbf7b5f0 2024-04-23T21:37:21.268 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:21.269+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc182ea0 2024-04-23T21:37:21.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 21533696 bytes 2024-04-23T21:37:23.864 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2024-04-23T21:37:23.864 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:37:23.865 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2024-04-23T21:37:23.865 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2024-04-23T21:37:23.865 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2024-04-23T21:37:23.868 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2024-04-23T21:37:23.868 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2024-04-23T21:37:27.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 107710464 bytes 2024-04-23T21:37:27.509 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:37:27.509+0000 7fe96ab64640 -1 bdev(0x55a5abf68e00 /var/lib/ceph/osd/cluster2-2/block) aio_submit retries 11 2024-04-23T21:37:27.511 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:37:27.511+0000 7fe967b5e640 -1 bdev(0x55a5abf68e00 /var/lib/ceph/osd/cluster2-2/block) aio_submit retries 11 2024-04-23T21:37:27.953 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:27.953+0000 7f9212057640 -1 reset not still connected to 0x5614de560000 2024-04-23T21:37:27.953 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:27.953+0000 7f9212057640 -1 reset not still connected to 0x5614e4b7a8f0 2024-04-23T21:37:27.953 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:27.953+0000 7f9212057640 -1 reset not still connected to 0x5614ebce6dd0 2024-04-23T21:37:27.953 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:27.953+0000 7f9212057640 -1 reset not still connected to 0x5614f24eb2b0 2024-04-23T21:37:27.953 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:27.953+0000 7f9212057640 -1 reset not still connected to 0x5614f603a9c0 2024-04-23T21:37:28.216 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:28.216+0000 7f32b7199640 -1 reset not still connected to 0x5617f67b68f0 2024-04-23T21:37:28.218 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:28.216+0000 7f32b7199640 -1 reset not still connected to 0x5617f727b2b0 2024-04-23T21:37:28.219 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:28.219+0000 7f32b7199640 -1 reset not still connected to 0x5618109795f0 2024-04-23T21:37:28.219 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:28.219+0000 7f32b7199640 -1 reset not still connected to 0x561816d2b6c0 2024-04-23T21:37:28.219 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:28.220+0000 7f32b7199640 -1 reset not still connected to 0x5618283af380 2024-04-23T21:37:29.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 129381376 bytes 2024-04-23T21:37:30.278 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:30.277+0000 7f9212057640 -1 reset not still connected to 0x5614de5295f0 2024-04-23T21:37:30.279 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:30.277+0000 7f9212057640 -1 reset not still connected to 0x5614e072a000 2024-04-23T21:37:30.279 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:30.277+0000 7f9212057640 -1 reset not still connected to 0x5614e35d1a00 2024-04-23T21:37:30.279 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:30.277+0000 7f9212057640 -1 reset not still connected to 0x5614ee548410 2024-04-23T21:37:30.279 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:30.277+0000 7f9212057640 -1 reset not still connected to 0x5614fc7ae410 2024-04-23T21:37:30.916 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:30.915+0000 7fc40b4f4640 -1 reset not still connected to 0x5640999bed00 2024-04-23T21:37:30.916 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:30.916+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4ed8f70 2024-04-23T21:37:30.916 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:30.916+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:37:30.919 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:30.919+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bbf7b5f0 2024-04-23T21:37:30.919 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:30.919+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc182ea0 2024-04-23T21:37:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 246838272 bytes 2024-04-23T21:37:32.399 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:32.399+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b1d38270 2024-04-23T21:37:32.399 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:32.399+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:37:32.399 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:32.399+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:37:33.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:37:33.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:37:33.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:37:33.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:37:33.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:37:33.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_35 2024-04-23T21:37:33.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_35 journal 128M 2024-04-23T21:37:33.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:37:33.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:33.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:33.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:33.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:37:33.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:33.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:37:33.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:37:33.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:33.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_35 128M 2024-04-23T21:37:33.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:37:33.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:33.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:33.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:33.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:37:33.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:33.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:37:33.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:37:33.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:37:33.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:33.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_35 2024-04-23T21:37:33.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:37:33.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:33.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:37:33.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:37:33.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:37:33.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:37:33.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_35 2024-04-23T21:37:33.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:37:33.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:33.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:37:33.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:37:33.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=13 2024-04-23T21:37:33.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:37:33.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_35 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:37:34.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 23860224 bytes 2024-04-23T21:37:35.206 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:35.206+0000 7f9212057640 -1 reset not still connected to 0x5614f36add40 2024-04-23T21:37:35.209 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:35.208+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:37:35.210 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:35.208+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:37:35.210 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:35.209+0000 7f9212057640 -1 reset not still connected to 0x561506553790 2024-04-23T21:37:35.211 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:35.211+0000 7f9212057640 -1 reset not still connected to 0x561506a85930 2024-04-23T21:37:36.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 42203136 bytes 2024-04-23T21:37:36.757 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:36.757+0000 7f32b7199640 -1 reset not still connected to 0x5618006e0270 2024-04-23T21:37:36.757 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:36.757+0000 7f32b7199640 -1 reset not still connected to 0x56180a12ba00 2024-04-23T21:37:36.758 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:36.758+0000 7f32b7199640 -1 reset not still connected to 0x56181119c680 2024-04-23T21:37:36.758 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:36.758+0000 7f32b7199640 -1 reset not still connected to 0x5618140d9110 2024-04-23T21:37:37.435 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:37.435+0000 7f32b7199640 -1 reset not still connected to 0x5618006e0270 2024-04-23T21:37:37.435 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:37.435+0000 7f32b7199640 -1 reset not still connected to 0x56180a12ba00 2024-04-23T21:37:37.435 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:37.435+0000 7f32b7199640 -1 reset not still connected to 0x56181119c680 2024-04-23T21:37:37.441 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:37.442+0000 7f32b7199640 -1 reset not still connected to 0x5618140d9110 2024-04-23T21:37:37.505 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:37.505+0000 7f32b7199640 -1 reset not still connected to 0x56181be82d00 2024-04-23T21:37:37.505 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:37.505+0000 7f32b7199640 -1 reset not still connected to 0x56181f915a00 2024-04-23T21:37:37.505 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:37.505+0000 7f32b7199640 -1 reset not still connected to 0x561827038a90 2024-04-23T21:37:37.524 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:37.524+0000 7f32b7199640 -1 reset not still connected to 0x561827038a90 2024-04-23T21:37:38.193 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:38.193+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a9049c70 2024-04-23T21:37:38.193 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:38.193+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ad4ce340 2024-04-23T21:37:38.193 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:38.193+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b1d38270 2024-04-23T21:37:38.193 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:38.193+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:37:38.193 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:38.193+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:37:40.949 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:40.948+0000 7f9212057640 -1 reset not still connected to 0x5614de601ad0 2024-04-23T21:37:40.950 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:40.950+0000 7f9212057640 -1 reset not still connected to 0x5614e5bb4b60 2024-04-23T21:37:40.950 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:40.950+0000 7f9212057640 -1 reset not still connected to 0x5614f36add40 2024-04-23T21:37:40.950 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:40.951+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:37:40.950 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:40.951+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:37:40.950 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:40.951+0000 7f9212057640 -1 reset not still connected to 0x561506553790 2024-04-23T21:37:40.950 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:40.951+0000 7f9212057640 -1 reset not still connected to 0x561506a85930 2024-04-23T21:37:41.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 153731072 bytes 2024-04-23T21:37:41.264 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:37:41.265+0000 7f5860e07640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 9 2024-04-23T21:37:41.267 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:37:41.268+0000 7f5861608640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 19 2024-04-23T21:37:42.488 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:42.489+0000 7f32b7199640 -1 reset not still connected to 0x56181be82d00 2024-04-23T21:37:42.489 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:42.489+0000 7f32b7199640 -1 reset not still connected to 0x56181f915a00 2024-04-23T21:37:42.489 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:42.489+0000 7f32b7199640 -1 reset not still connected to 0x561827038a90 2024-04-23T21:37:46.064 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:46.064+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c12d7790 2024-04-23T21:37:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:37:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:37:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:37:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:37:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:37:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_36 2024-04-23T21:37:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_36 journal 128M 2024-04-23T21:37:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:37:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:46.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:46.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:46.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:37:46.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:46.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:37:46.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:37:46.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:46.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_36 128M 2024-04-23T21:37:46.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:37:46.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:46.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:46.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:46.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:37:46.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:46.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:37:46.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:37:46.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:37:46.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:46.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_36 2024-04-23T21:37:46.327 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:46.325+0000 7fc40b4f4640 -1 reset not still connected to 0x564096727450 2024-04-23T21:37:46.327 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:46.325+0000 7fc40b4f4640 -1 reset not still connected to 0x56409e1b3c70 2024-04-23T21:37:46.327 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:46.325+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a524ca90 2024-04-23T21:37:46.327 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:46.325+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aad21c70 2024-04-23T21:37:46.327 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:46.326+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bbeccf70 2024-04-23T21:37:46.327 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:37:46.326+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c12d7790 2024-04-23T21:37:46.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:37:46.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:46.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:37:46.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:37:46.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:37:46.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:37:46.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_36 2024-04-23T21:37:46.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:37:46.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:46.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:37:46.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:37:46.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=11 2024-04-23T21:37:46.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:37:46.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_36 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:37:48.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 19266560 bytes 2024-04-23T21:37:49.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 39490560 bytes 2024-04-23T21:37:51.334 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:51.334+0000 7f9212057640 -1 reset not still connected to 0x5614dddbf2b0 2024-04-23T21:37:51.335 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:51.335+0000 7f9212057640 -1 reset not still connected to 0x5614de5295f0 2024-04-23T21:37:51.335 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:51.335+0000 7f9212057640 -1 reset not still connected to 0x5614e35d1a00 2024-04-23T21:37:51.335 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:51.335+0000 7f9212057640 -1 reset not still connected to 0x5614ee548410 2024-04-23T21:37:51.335 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:51.335+0000 7f9212057640 -1 reset not still connected to 0x5614fc7ae410 2024-04-23T21:37:51.593 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:51.589+0000 7f9212057640 -1 reset not still connected to 0x5614d471d790 2024-04-23T21:37:52.946 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:52.945+0000 7f32b7199640 -1 reset not still connected to 0x56180a12ba00 2024-04-23T21:37:52.946 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:52.945+0000 7f32b7199640 -1 reset not still connected to 0x56181119c680 2024-04-23T21:37:52.946 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:37:52.945+0000 7f32b7199640 -1 reset not still connected to 0x5618140d9110 2024-04-23T21:37:54.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 121364480 bytes 2024-04-23T21:37:55.762 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:37:55.763+0000 7f5866612640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 23 2024-04-23T21:37:55.764 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:37:55.765+0000 7f5861608640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 33 2024-04-23T21:37:55.904 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:55.904+0000 7f9212057640 -1 reset not still connected to 0x5614d8ec2820 2024-04-23T21:37:55.904 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:55.904+0000 7f9212057640 -1 reset not still connected to 0x5614d9bd9790 2024-04-23T21:37:55.904 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:55.904+0000 7f9212057640 -1 reset not still connected to 0x5614ddd505b0 2024-04-23T21:37:55.907 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:55.907+0000 7f9212057640 -1 reset not still connected to 0x5614de3b0b60 2024-04-23T21:37:55.907 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:55.907+0000 7f9212057640 -1 reset not still connected to 0x5614de3b1380 2024-04-23T21:37:55.907 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:55.907+0000 7f9212057640 -1 reset not still connected to 0x5614de560000 2024-04-23T21:37:55.907 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:55.907+0000 7f9212057640 -1 reset not still connected to 0x5614e4b7a8f0 2024-04-23T21:37:55.907 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:55.907+0000 7f9212057640 -1 reset not still connected to 0x5614ebce6dd0 2024-04-23T21:37:55.907 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:55.907+0000 7f9212057640 -1 reset not still connected to 0x5614f24eb2b0 2024-04-23T21:37:55.908 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:55.907+0000 7f9212057640 -1 reset not still connected to 0x5614f603a9c0 2024-04-23T21:37:55.916 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:55.915+0000 7f9212057640 -1 reset not still connected to 0x5614ddd505b0 2024-04-23T21:37:55.916 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:55.915+0000 7f9212057640 -1 reset not still connected to 0x5614de3b1380 2024-04-23T21:37:55.916 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:55.915+0000 7f9212057640 -1 reset not still connected to 0x5614de560000 2024-04-23T21:37:55.916 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:55.916+0000 7f9212057640 -1 reset not still connected to 0x5614e4b7a8f0 2024-04-23T21:37:55.917 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:55.916+0000 7f9212057640 -1 reset not still connected to 0x5614ebce6dd0 2024-04-23T21:37:55.917 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:55.916+0000 7f9212057640 -1 reset not still connected to 0x5614f24eb2b0 2024-04-23T21:37:55.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 147567616 bytes 2024-04-23T21:37:57.257 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:57.256+0000 7f9212057640 -1 reset not still connected to 0x5614ee548410 2024-04-23T21:37:57.257 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:37:57.256+0000 7f9212057640 -1 reset not still connected to 0x5614fc7ae410 2024-04-23T21:37:57.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:37:57.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:37:57.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:37:57.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:37:57.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:37:57.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_37 2024-04-23T21:37:57.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_37 journal 128M 2024-04-23T21:37:57.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:37:57.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:57.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:57.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:57.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:37:57.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:57.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:37:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:37:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_37 128M 2024-04-23T21:37:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:37:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:37:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:37:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:37:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:37:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:37:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_37 2024-04-23T21:37:57.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:37:57.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:57.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:37:57.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:37:57.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:37:57.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:37:57.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_37 2024-04-23T21:37:57.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:37:57.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:37:57.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:37:57.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:37:57.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=8 2024-04-23T21:37:57.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:37:57.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_37 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:38:00.259 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:00.259+0000 7f9212057640 -1 reset not still connected to 0x5614ee548410 2024-04-23T21:38:00.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 43076608 bytes 2024-04-23T21:38:02.525 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:02.525+0000 7f9212057640 -1 reset not still connected to 0x5614f36add40 2024-04-23T21:38:02.529 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:02.529+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:38:02.529 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:02.529+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:38:02.529 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:02.529+0000 7f9212057640 -1 reset not still connected to 0x561506553790 2024-04-23T21:38:02.529 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:02.530+0000 7f9212057640 -1 reset not still connected to 0x561506a85930 2024-04-23T21:38:02.594 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:02.593+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:38:02.599 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:02.599+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:38:02.599 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:02.599+0000 7fc40b4f4640 -1 reset not still connected to 0x564084c61380 2024-04-23T21:38:02.599 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:02.599+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a9049c70 2024-04-23T21:38:02.599 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:02.599+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:38:02.599 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:02.599+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:38:03.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 76725248 bytes 2024-04-23T21:38:03.257 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:03.257+0000 7f9212057640 -1 reset not still connected to 0x5614f36add40 2024-04-23T21:38:03.260 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:03.261+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:38:03.260 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:03.261+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:38:03.264 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:03.264+0000 7f9212057640 -1 reset not still connected to 0x561506553790 2024-04-23T21:38:03.264 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:03.264+0000 7f9212057640 -1 reset not still connected to 0x561506a85930 2024-04-23T21:38:03.308 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:38:03.309+0000 7f83ea356640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 1 2024-04-23T21:38:03.552 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:03.552+0000 7f9212057640 -1 reset not still connected to 0x5614de601ad0 2024-04-23T21:38:03.554 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:03.555+0000 7f9212057640 -1 reset not still connected to 0x5614e5bb4b60 2024-04-23T21:38:03.555 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:03.555+0000 7f9212057640 -1 reset not still connected to 0x5614f36add40 2024-04-23T21:38:03.555 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:03.555+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:38:03.555 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:03.555+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:38:03.558 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:03.558+0000 7f9212057640 -1 reset not still connected to 0x561506553790 2024-04-23T21:38:03.558 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:03.558+0000 7f9212057640 -1 reset not still connected to 0x561506a85930 2024-04-23T21:38:04.989 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:04.990+0000 7f9212057640 -1 reset not still connected to 0x5614f36add40 2024-04-23T21:38:04.990 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:04.990+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:38:04.990 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:04.990+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:38:04.990 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:04.990+0000 7f9212057640 -1 reset not still connected to 0x561506553790 2024-04-23T21:38:04.990 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:04.990+0000 7f9212057640 -1 reset not still connected to 0x561506a85930 2024-04-23T21:38:05.021 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:05.022+0000 7f9212057640 -1 reset not still connected to 0x5614ebce6dd0 2024-04-23T21:38:05.022 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:05.022+0000 7f9212057640 -1 reset not still connected to 0x5614f24eb2b0 2024-04-23T21:38:05.023 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:05.023+0000 7f9212057640 -1 reset not still connected to 0x5614f603a9c0 2024-04-23T21:38:05.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:38:05.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:38:05.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:38:05.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:38:05.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:38:05.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_38 2024-04-23T21:38:05.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_38 journal 128M 2024-04-23T21:38:05.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:05.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:05.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:05.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:05.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:38:05.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:05.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:38:05.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:38:05.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:05.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_38 128M 2024-04-23T21:38:05.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:05.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:05.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:05.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:05.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:38:05.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:05.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:38:05.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:38:05.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:38:05.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:05.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_38 2024-04-23T21:38:05.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:38:05.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:38:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:38:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:38:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:38:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_38 2024-04-23T21:38:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:38:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:38:05.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=9 2024-04-23T21:38:05.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:38:05.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_38 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:38:07.534 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:07.534+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a9049c70 2024-04-23T21:38:07.534 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:07.534+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ad4ce340 2024-04-23T21:38:07.534 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:07.534+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b1d38270 2024-04-23T21:38:07.534 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:07.534+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:38:07.534 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:07.534+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:38:08.110 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:08.110+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4610820 2024-04-23T21:38:08.110 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:08.111+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ad4ce340 2024-04-23T21:38:08.111 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:08.111+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b1d38270 2024-04-23T21:38:08.111 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:08.111+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:38:08.111 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:08.111+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:38:11.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 77281280 bytes 2024-04-23T21:38:13.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:38:13.409+0000 7f147384e640 -1 rbd::mirror::image_replayer::journal::Replayer: 0x563a4e6b6e00 get_replay_status: replay not running 2024-04-23T21:38:13.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 95551488 bytes 2024-04-23T21:38:13.907 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:38:13.907+0000 7fe97bfc2640 -1 reset not still connected to 0x55a5b21cdc70 2024-04-23T21:38:13.908 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:38:13.907+0000 7fe97bfc2640 -1 reset not still connected to 0x55a5d5632a90 2024-04-23T21:38:13.908 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:38:13.907+0000 7fe97bfc2640 -1 reset not still connected to 0x55a5d82b95f0 2024-04-23T21:38:13.908 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:38:13.908+0000 7fe97bfc2640 -1 reset not still connected to 0x55a5e74b5e10 2024-04-23T21:38:14.113 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:38:14.111+0000 7f32b7199640 -1 reset not still connected to 0x561803e01c70 2024-04-23T21:38:14.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:38:14.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:38:14.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:38:14.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:38:14.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:38:14.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_39 2024-04-23T21:38:14.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_39 journal 128M 2024-04-23T21:38:14.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:14.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:14.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:14.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:14.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:38:14.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:14.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:38:14.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:38:14.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:14.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_39 128M 2024-04-23T21:38:14.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:14.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:14.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:14.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:14.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:38:14.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:14.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:38:14.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:38:14.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:38:14.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:14.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_39 2024-04-23T21:38:14.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:38:14.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:14.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:38:14.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:38:14.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:38:14.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:38:14.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_39 2024-04-23T21:38:14.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:14.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:14.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:38:14.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:38:14.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=8 2024-04-23T21:38:14.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:38:14.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_39 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:38:17.925 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:17.924+0000 7fc40b4f4640 -1 reset not still connected to 0x564089e3ac30 2024-04-23T21:38:17.925 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:17.924+0000 7fc40b4f4640 -1 reset not still connected to 0x564090b145b0 2024-04-23T21:38:17.925 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:17.924+0000 7fc40b4f4640 -1 reset not still connected to 0x564099bba1a0 2024-04-23T21:38:17.925 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:17.924+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4610820 2024-04-23T21:38:17.925 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:17.924+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a9049c70 2024-04-23T21:38:17.925 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:17.924+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b1d38270 2024-04-23T21:38:17.926 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:17.926+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:38:17.926 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:17.926+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:38:18.431 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:18.431+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b3bdb1e0 2024-04-23T21:38:18.431 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:18.431+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bbeccf70 2024-04-23T21:38:18.431 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:18.431+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c12d7790 2024-04-23T21:38:18.435 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:18.436+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c18e0d00 2024-04-23T21:38:19.510 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:38:19.509+0000 7fe97bfc2640 -1 reset not still connected to 0x55a5dcd65d40 2024-04-23T21:38:19.513 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:38:19.513+0000 7fe97bfc2640 -1 reset not still connected to 0x55a5ddd296c0 2024-04-23T21:38:19.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 66798592 bytes 2024-04-23T21:38:22.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:38:22.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:38:22.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:38:22.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:38:22.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:38:22.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_40 2024-04-23T21:38:22.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_40 journal 128M 2024-04-23T21:38:22.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:22.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:22.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:22.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:22.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:38:22.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:22.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:38:22.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:38:22.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:22.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_40 128M 2024-04-23T21:38:22.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:22.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:22.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:22.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:22.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:38:22.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:22.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:38:22.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:38:22.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:38:22.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:22.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_40 2024-04-23T21:38:22.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:38:22.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:22.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:38:22.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:38:22.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:38:22.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:38:22.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_40 2024-04-23T21:38:22.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:22.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:22.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:38:22.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:38:22.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=12 2024-04-23T21:38:22.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:38:22.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_40 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:38:23.780 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:38:23.780+0000 7f32b7199640 -1 reset not still connected to 0x561803e01c70 2024-04-23T21:38:24.432 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:24.430+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:38:24.433 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:24.432+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:38:25.060 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:25.060+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:38:25.060 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:25.060+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aef23380 2024-04-23T21:38:27.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 84286464 bytes 2024-04-23T21:38:28.442 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:38:28.442+0000 7f32b7199640 -1 reset not still connected to 0x56181be82d00 2024-04-23T21:38:28.442 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:38:28.442+0000 7f32b7199640 -1 reset not still connected to 0x56181f915a00 2024-04-23T21:38:28.442 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:38:28.442+0000 7f32b7199640 -1 reset not still connected to 0x561827038a90 2024-04-23T21:38:29.547 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:38:29.547+0000 7f32b7199640 -1 reset not still connected to 0x5617ee64dad0 2024-04-23T21:38:29.547 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:38:29.547+0000 7f32b7199640 -1 reset not still connected to 0x561803e01c70 2024-04-23T21:38:29.547 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:38:29.547+0000 7f32b7199640 -1 reset not still connected to 0x5618279e84e0 2024-04-23T21:38:30.421 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:30.421+0000 7fc40b4f4640 -1 reset not still connected to 0x56408b1c1a00 2024-04-23T21:38:30.421 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:30.421+0000 7fc40b4f4640 -1 reset not still connected to 0x56408d04f110 2024-04-23T21:38:30.421 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:30.421+0000 7fc40b4f4640 -1 reset not still connected to 0x56409dcfb930 2024-04-23T21:38:30.421 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:30.421+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac2d4dd0 2024-04-23T21:38:30.421 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:30.421+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b3910ea0 2024-04-23T21:38:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 134109184 bytes 2024-04-23T21:38:31.416 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:31.414+0000 7fc40b4f4640 -1 reset not still connected to 0x5640839d5860 2024-04-23T21:38:31.419 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:31.419+0000 7fc40b4f4640 -1 reset not still connected to 0x5640879b6000 2024-04-23T21:38:31.419 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:31.419+0000 7fc40b4f4640 -1 reset not still connected to 0x564089fdf6c0 2024-04-23T21:38:31.419 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:31.419+0000 7fc40b4f4640 -1 reset not still connected to 0x56408b1c1a00 2024-04-23T21:38:31.419 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:31.419+0000 7fc40b4f4640 -1 reset not still connected to 0x56408d04f110 2024-04-23T21:38:31.419 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:31.419+0000 7fc40b4f4640 -1 reset not still connected to 0x56409dcfb930 2024-04-23T21:38:31.419 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:31.419+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac2d4dd0 2024-04-23T21:38:31.419 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:31.419+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b3910ea0 2024-04-23T21:38:31.420 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:31.419+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b39904e0 2024-04-23T21:38:33.176 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:38:33.177+0000 7f32b7199640 -1 reset not still connected to 0x5618140d9110 2024-04-23T21:38:34.356 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:34.354+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ad4ce340 2024-04-23T21:38:34.358 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:34.355+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b1d38270 2024-04-23T21:38:34.358 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:34.355+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b28c95f0 2024-04-23T21:38:34.358 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:34.355+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:38:34.358 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:34.356+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:38:34.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:38:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:38:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:38:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:38:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:38:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_41 2024-04-23T21:38:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_41 journal 128M 2024-04-23T21:38:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:38:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:38:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:38:34.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_41 128M 2024-04-23T21:38:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:38:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:38:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:38:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:38:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:34.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_41 2024-04-23T21:38:34.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:38:34.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:34.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:38:34.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:38:34.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:38:34.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:38:34.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_41 2024-04-23T21:38:34.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:34.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:34.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:38:34.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:38:34.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=7 2024-04-23T21:38:34.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:38:34.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_41 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:38:37.288 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:37.287+0000 7fc40b4f4640 -1 reset not still connected to 0x5640846be410 2024-04-23T21:38:37.288 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:37.287+0000 7fc40b4f4640 -1 reset not still connected to 0x56408576eb60 2024-04-23T21:38:37.288 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:37.287+0000 7fc40b4f4640 -1 reset not still connected to 0x564085f0eb60 2024-04-23T21:38:37.288 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:37.287+0000 7fc40b4f4640 -1 reset not still connected to 0x56408b23b520 2024-04-23T21:38:37.288 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:37.287+0000 7fc40b4f4640 -1 reset not still connected to 0x5640999bed00 2024-04-23T21:38:37.289 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:37.289+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a114be10 2024-04-23T21:38:37.289 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:37.289+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4ed8f70 2024-04-23T21:38:37.289 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:37.289+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:38:37.289 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:37.289+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aef23380 2024-04-23T21:38:37.289 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:37.289+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bbf7b5f0 2024-04-23T21:38:37.289 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:37.289+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc182ea0 2024-04-23T21:38:38.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 38863872 bytes 2024-04-23T21:38:40.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 66324480 bytes 2024-04-23T21:38:41.757 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:41.757+0000 7fc40b4f4640 -1 reset not still connected to 0x56408b1c1a00 2024-04-23T21:38:41.760 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:41.760+0000 7fc40b4f4640 -1 reset not still connected to 0x56409dcfb930 2024-04-23T21:38:41.760 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:41.760+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac2d4dd0 2024-04-23T21:38:41.760 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:41.760+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b3910ea0 2024-04-23T21:38:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:38:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:38:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:38:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:38:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:38:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_42 2024-04-23T21:38:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_42 journal 128M 2024-04-23T21:38:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:38:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:38:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:38:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_42 128M 2024-04-23T21:38:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:38:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:38:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:38:41.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:38:41.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:41.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_42 2024-04-23T21:38:41.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:38:41.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:41.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:38:41.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:38:41.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:38:41.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:38:41.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_42 2024-04-23T21:38:41.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:41.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:41.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:38:41.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:38:41.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=12 2024-04-23T21:38:41.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:38:41.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_42 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:38:43.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 29694976 bytes 2024-04-23T21:38:44.172 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9661568878966899) > weight (0.25) 2024-04-23T21:38:44.172 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7821446390075979) > weight (0.25) 2024-04-23T21:38:44.172 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8947114872882355) > weight (0.25) 2024-04-23T21:38:44.172 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2024-04-23T21:38:44.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2024-04-23T21:38:44.173 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2024-04-23T21:38:44.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:38:44.175+0000 7feb43449640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 35434) UID: 1000 2024-04-23T21:38:44.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:38:44.251+0000 7feb3bc3a640 -1 rbd::mirror::image_replayer::journal::Replayer: 0x56512dbea000 handle_process_entry_safe: failed to commit journal event: (85) Interrupted system call should be restarted 2024-04-23T21:38:44.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:38:44.251+0000 7feb3bc3a640 -1 rbd::mirror::image_replayer::journal::Replayer: 0x56512dbea000 handle_replay_complete: replay encountered an error: (85) Interrupted system call should be restarted 2024-04-23T21:38:45.174 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2024-04-23T21:38:45.174 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:38:45.916 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:45.916+0000 7fc40b4f4640 -1 reset not still connected to 0x564089fdf6c0 2024-04-23T21:38:45.917 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:45.916+0000 7fc40b4f4640 -1 reset not still connected to 0x56408b1c1a00 2024-04-23T21:38:45.917 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:45.916+0000 7fc40b4f4640 -1 reset not still connected to 0x56408d04f110 2024-04-23T21:38:45.917 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:45.916+0000 7fc40b4f4640 -1 reset not still connected to 0x56409dcfb930 2024-04-23T21:38:45.917 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:45.916+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a97caea0 2024-04-23T21:38:45.917 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:45.916+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac2d4dd0 2024-04-23T21:38:45.917 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:38:45.916+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b3910ea0 2024-04-23T21:38:47.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 97667072 bytes 2024-04-23T21:38:47.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 98263040 bytes 2024-04-23T21:38:48.869 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2024-04-23T21:38:48.869 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2024-04-23T21:38:48.869 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2024-04-23T21:38:48.870 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:38:48.871+0000 7f4e6158e640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 61991) UID: 1000 2024-04-23T21:38:50.588 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:38:50.589+0000 7f83eb358640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 15 2024-04-23T21:38:50.588 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:38:50.589+0000 7f83ebb59640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 16 2024-04-23T21:38:50.589 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:38:50.591+0000 7f83ed35c640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 15 2024-04-23T21:38:50.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 134261760 bytes 2024-04-23T21:38:51.275 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2024-04-23T21:38:51.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2024-04-23T21:38:51.276 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2024-04-23T21:38:51.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2024-04-23T21:38:51.278 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2024-04-23T21:38:51.632 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:51.630+0000 7f9212057640 -1 reset not still connected to 0x5614f36add40 2024-04-23T21:38:51.632 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:51.631+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:38:51.632 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:51.631+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:38:51.632 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:51.631+0000 7f9212057640 -1 reset not still connected to 0x561506553790 2024-04-23T21:38:51.633 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:51.632+0000 7f9212057640 -1 reset not still connected to 0x561506a85930 2024-04-23T21:38:52.709 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:52.710+0000 7f9212057640 -1 reset not still connected to 0x5614e5bb4b60 2024-04-23T21:38:52.710 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:52.710+0000 7f9212057640 -1 reset not still connected to 0x5614f36add40 2024-04-23T21:38:52.710 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:52.710+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:38:52.710 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:52.710+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:38:52.710 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:52.710+0000 7f9212057640 -1 reset not still connected to 0x561506553790 2024-04-23T21:38:52.710 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:52.710+0000 7f9212057640 -1 reset not still connected to 0x561506a85930 2024-04-23T21:38:52.870 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2024-04-23T21:38:52.870 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:38:52.870 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2024-04-23T21:38:52.871 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2024-04-23T21:38:52.871 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2024-04-23T21:38:52.873 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2024-04-23T21:38:52.873 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2024-04-23T21:38:53.818 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:53.819+0000 7f9212057640 -1 reset not still connected to 0x5614de3b1380 2024-04-23T21:38:53.819 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:53.820+0000 7f9212057640 -1 reset not still connected to 0x5614de560000 2024-04-23T21:38:53.819 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:53.820+0000 7f9212057640 -1 reset not still connected to 0x5614e4b7a8f0 2024-04-23T21:38:53.819 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:53.820+0000 7f9212057640 -1 reset not still connected to 0x5614ebce6dd0 2024-04-23T21:38:53.819 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:53.820+0000 7f9212057640 -1 reset not still connected to 0x5614f24eb2b0 2024-04-23T21:38:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:38:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:38:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:38:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:38:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:38:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_43 2024-04-23T21:38:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_43 journal 128M 2024-04-23T21:38:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:38:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:38:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:38:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_43 128M 2024-04-23T21:38:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:38:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:38:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:38:53.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:38:53.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:38:53.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_43 2024-04-23T21:38:53.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:38:53.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:53.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:38:53.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:38:53.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:38:53.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:38:53.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_43 2024-04-23T21:38:53.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:38:53.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:38:53.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:38:53.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:38:53.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=7 2024-04-23T21:38:53.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:38:53.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_43 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:38:58.960 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:58.959+0000 7f9212057640 -1 reset not still connected to 0x5614d7fb6a90 2024-04-23T21:38:58.968 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:58.969+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:38:59.392 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:59.392+0000 7f9212057640 -1 reset not still connected to 0x5614ee548410 2024-04-23T21:38:59.392 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:59.392+0000 7f9212057640 -1 reset not still connected to 0x5614fc7ae410 2024-04-23T21:38:59.686 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:59.686+0000 7f9212057640 -1 reset not still connected to 0x5614d36901a0 2024-04-23T21:38:59.686 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:59.686+0000 7f9212057640 -1 reset not still connected to 0x5614d7fb6a90 2024-04-23T21:38:59.689 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:59.690+0000 7f9212057640 -1 reset not still connected to 0x5614de601ad0 2024-04-23T21:38:59.689 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:59.690+0000 7f9212057640 -1 reset not still connected to 0x5614f36add40 2024-04-23T21:38:59.689 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:59.690+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:38:59.690 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:59.690+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:38:59.690 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:59.690+0000 7f9212057640 -1 reset not still connected to 0x561506553790 2024-04-23T21:38:59.692 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:38:59.692+0000 7f9212057640 -1 reset not still connected to 0x561506a85930 2024-04-23T21:38:59.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 78759936 bytes 2024-04-23T21:39:00.409 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:00.408+0000 7f9212057640 -1 reset not still connected to 0x5614ebce6dd0 2024-04-23T21:39:00.409 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:00.409+0000 7f9212057640 -1 reset not still connected to 0x5614f24eb2b0 2024-04-23T21:39:00.409 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:00.409+0000 7f9212057640 -1 reset not still connected to 0x5614f603a9c0 2024-04-23T21:39:00.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:39:00.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:39:00.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:39:00.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:39:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:39:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_44 2024-04-23T21:39:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_44 journal 128M 2024-04-23T21:39:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:39:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:39:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:39:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_44 128M 2024-04-23T21:39:00.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:39:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:39:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:39:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:39:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:00.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_44 2024-04-23T21:39:01.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:39:01.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:01.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:39:01.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:39:01.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:39:01.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:39:01.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_44 2024-04-23T21:39:01.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:01.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:01.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:39:01.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:39:01.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=5 2024-04-23T21:39:01.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:39:01.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_44 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:39:01.843 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:39:01.843+0000 7f83fbfb5640 -1 reset not still connected to 0x562a556c4270 2024-04-23T21:39:06.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:39:06.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:39:06.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:39:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:39:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:39:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_45 2024-04-23T21:39:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_45 journal 128M 2024-04-23T21:39:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:39:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:39:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:39:06.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_45 128M 2024-04-23T21:39:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:39:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:39:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:39:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:39:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:06.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_45 2024-04-23T21:39:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:39:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:39:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:39:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:39:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:39:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_45 2024-04-23T21:39:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:06.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:39:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:39:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=13 2024-04-23T21:39:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:39:06.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_45 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:39:07.799 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:07.799+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a97caea0 2024-04-23T21:39:07.799 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:07.799+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac2d4dd0 2024-04-23T21:39:07.799 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:07.799+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b3910ea0 2024-04-23T21:39:07.799 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:07.799+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b39904e0 2024-04-23T21:39:07.883 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:07.884+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4ed8f70 2024-04-23T21:39:07.884 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:07.884+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:39:07.884 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:07.884+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aef23380 2024-04-23T21:39:07.887 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:07.888+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bbf7b5f0 2024-04-23T21:39:08.244 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:08.244+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a97caea0 2024-04-23T21:39:08.245 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:08.244+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac2d4dd0 2024-04-23T21:39:08.245 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:08.244+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b3910ea0 2024-04-23T21:39:08.246 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:08.247+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b39904e0 2024-04-23T21:39:08.418 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:08.418+0000 7fc40b4f4640 -1 reset not still connected to 0x56409dcfb930 2024-04-23T21:39:08.423 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:08.424+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a97caea0 2024-04-23T21:39:08.424 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:08.424+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac2d4dd0 2024-04-23T21:39:08.424 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:08.424+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b3910ea0 2024-04-23T21:39:08.424 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:08.424+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b39904e0 2024-04-23T21:39:08.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:39:08.510+0000 7f382edc3640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd85ccb40 finish: resending after timeout 2024-04-23T21:39:09.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 44741632 bytes 2024-04-23T21:39:09.792 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:09.790+0000 7f32b7199640 -1 reset not still connected to 0x5617fa760680 2024-04-23T21:39:09.793 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:09.793+0000 7f32b7199640 -1 reset not still connected to 0x561804f58750 2024-04-23T21:39:09.793 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:09.793+0000 7f32b7199640 -1 reset not still connected to 0x5618067e4d00 2024-04-23T21:39:09.794 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:09.793+0000 7f32b7199640 -1 reset not still connected to 0x5618109795f0 2024-04-23T21:39:09.794 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:09.793+0000 7f32b7199640 -1 reset not still connected to 0x56181105c9c0 2024-04-23T21:39:09.794 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:09.793+0000 7f32b7199640 -1 reset not still connected to 0x561816d2b6c0 2024-04-23T21:39:09.794 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:09.793+0000 7f32b7199640 -1 reset not still connected to 0x56182326f860 2024-04-23T21:39:09.794 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:09.793+0000 7f32b7199640 -1 reset not still connected to 0x5618275f4750 2024-04-23T21:39:09.794 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:09.794+0000 7f32b7199640 -1 reset not still connected to 0x5618283af380 2024-04-23T21:39:10.540 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:10.540+0000 7f32b7199640 -1 reset not still connected to 0x5617ff7956c0 2024-04-23T21:39:10.540 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:10.540+0000 7f32b7199640 -1 reset not still connected to 0x5618006e0270 2024-04-23T21:39:10.540 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:10.540+0000 7f32b7199640 -1 reset not still connected to 0x56180a12ba00 2024-04-23T21:39:10.540 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:10.540+0000 7f32b7199640 -1 reset not still connected to 0x56181119c680 2024-04-23T21:39:10.540 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:10.540+0000 7f32b7199640 -1 reset not still connected to 0x5618140d9110 2024-04-23T21:39:10.865 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:10.865+0000 7f32b7199640 -1 reset not still connected to 0x5617f67b68f0 2024-04-23T21:39:10.865 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:10.865+0000 7f32b7199640 -1 reset not still connected to 0x5617f727b2b0 2024-04-23T21:39:10.865 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:10.865+0000 7f32b7199640 -1 reset not still connected to 0x5617fa760680 2024-04-23T21:39:10.865 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:10.865+0000 7f32b7199640 -1 reset not still connected to 0x561804f58750 2024-04-23T21:39:10.865 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:10.865+0000 7f32b7199640 -1 reset not still connected to 0x5618067e4d00 2024-04-23T21:39:10.865 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:10.866+0000 7f32b7199640 -1 reset not still connected to 0x5618109795f0 2024-04-23T21:39:10.866 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:10.866+0000 7f32b7199640 -1 reset not still connected to 0x56181105c9c0 2024-04-23T21:39:10.866 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:10.866+0000 7f32b7199640 -1 reset not still connected to 0x561816d2b6c0 2024-04-23T21:39:10.866 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:10.866+0000 7f32b7199640 -1 reset not still connected to 0x56182326f860 2024-04-23T21:39:10.866 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:10.866+0000 7f32b7199640 -1 reset not still connected to 0x5618275f4750 2024-04-23T21:39:10.868 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:10.868+0000 7f32b7199640 -1 reset not still connected to 0x5618283af380 2024-04-23T21:39:12.013 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:12.012+0000 7f32b7199640 -1 reset not still connected to 0x5618283af380 2024-04-23T21:39:13.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 132884480 bytes 2024-04-23T21:39:15.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:39:15.795+0000 7f99a36b6640 -1 rbd::mirror::image_replayer::journal::Replayer: 0x5619b95f2a80 get_replay_status: replay not running 2024-04-23T21:39:16.043 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:16.042+0000 7fc40b4f4640 -1 reset not still connected to 0x56408b23b520 2024-04-23T21:39:16.043 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:16.042+0000 7fc40b4f4640 -1 reset not still connected to 0x56408ba7a8f0 2024-04-23T21:39:16.043 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:16.042+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a114be10 2024-04-23T21:39:16.043 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:16.042+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4ed8f70 2024-04-23T21:39:16.043 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:16.042+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:39:16.043 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:16.042+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aef23380 2024-04-23T21:39:16.043 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:16.042+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bbf7b5f0 2024-04-23T21:39:16.354 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:39:16.354+0000 7f83ea356640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 4 2024-04-23T21:39:16.362 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:39:16.362+0000 7f83e9354640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 4 2024-04-23T21:39:16.646 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:39:16.646+0000 7fe97bfc2640 -1 reset not still connected to 0x55a5d81a76c0 2024-04-23T21:39:17.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:39:17.153+0000 7f382edc3640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd85ccb40 finish: resending after timeout 2024-04-23T21:39:17.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 203914240 bytes 2024-04-23T21:39:19.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:39:19.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:39:19.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:39:19.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:39:19.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:39:19.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_46 2024-04-23T21:39:19.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_46 journal 128M 2024-04-23T21:39:19.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:19.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:19.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:19.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:19.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:39:19.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:19.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:39:19.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:39:19.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:19.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_46 128M 2024-04-23T21:39:19.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:19.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:19.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:19.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:19.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:39:19.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:19.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:39:19.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:39:19.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:39:19.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:19.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_46 2024-04-23T21:39:19.202 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.200+0000 7fc40b4f4640 -1 reset not still connected to 0x56409dcfb930 2024-04-23T21:39:19.206 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.204+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a97caea0 2024-04-23T21:39:19.206 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.204+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ab374c30 2024-04-23T21:39:19.206 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.204+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aba0d450 2024-04-23T21:39:19.206 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.204+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac2d4dd0 2024-04-23T21:39:19.206 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.204+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b3910ea0 2024-04-23T21:39:19.206 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.204+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b39904e0 2024-04-23T21:39:19.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:39:19.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:19.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:39:19.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:39:19.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:39:19.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:39:19.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_46 2024-04-23T21:39:19.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:19.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:19.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:39:19.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:39:19.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=7 2024-04-23T21:39:19.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:39:19.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_46 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:39:19.587 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.587+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b28c8820 2024-04-23T21:39:19.587 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.587+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b28c95f0 2024-04-23T21:39:19.587 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.587+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:39:19.587 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.587+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:39:19.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 12946432 bytes 2024-04-23T21:39:19.829 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.829+0000 7fc40b4f4640 -1 reset not still connected to 0x564084c61380 2024-04-23T21:39:19.829 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.829+0000 7fc40b4f4640 -1 reset not still connected to 0x564090b145b0 2024-04-23T21:39:19.829 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.829+0000 7fc40b4f4640 -1 reset not still connected to 0x564099bba1a0 2024-04-23T21:39:19.833 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.831+0000 7fc40b4f4640 -1 reset not still connected to 0x56409c5f7e10 2024-04-23T21:39:19.833 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.831+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4610820 2024-04-23T21:39:19.833 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.831+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ad4ce340 2024-04-23T21:39:19.834 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.834+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:39:19.836 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:19.834+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:39:20.972 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:20.972+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:39:21.684 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:21.684+0000 7f9212057640 -1 reset not still connected to 0x5614dcb0ad00 2024-04-23T21:39:21.684 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:21.684+0000 7f9212057640 -1 reset not still connected to 0x5614de601ad0 2024-04-23T21:39:21.689 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:21.685+0000 7f9212057640 -1 reset not still connected to 0x5614f36add40 2024-04-23T21:39:21.690 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:21.688+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:39:21.690 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:21.688+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:39:21.690 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:21.688+0000 7f9212057640 -1 reset not still connected to 0x561506553790 2024-04-23T21:39:21.691 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:21.688+0000 7f9212057640 -1 reset not still connected to 0x561506a85930 2024-04-23T21:39:21.691 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:21.690+0000 7f9212057640 -1 reset not still connected to 0x56150e4685b0 2024-04-23T21:39:21.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 52049920 bytes 2024-04-23T21:39:22.433 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:22.433+0000 7f9212057640 -1 reset not still connected to 0x5614de601ad0 2024-04-23T21:39:22.433 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:22.433+0000 7f9212057640 -1 reset not still connected to 0x5614f36add40 2024-04-23T21:39:22.433 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:22.433+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:39:22.433 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:22.433+0000 7f9212057640 -1 reset not still connected to 0x561506553790 2024-04-23T21:39:22.440 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:22.441+0000 7f9212057640 -1 reset not still connected to 0x56150e4685b0 2024-04-23T21:39:22.663 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:22.660+0000 7f9212057640 -1 reset not still connected to 0x5614d298b380 2024-04-23T21:39:22.667 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:22.667+0000 7f9212057640 -1 reset not still connected to 0x5614dbf71790 2024-04-23T21:39:22.667 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:22.667+0000 7f9212057640 -1 reset not still connected to 0x5614e731a5b0 2024-04-23T21:39:22.668 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:22.667+0000 7f9212057640 -1 reset not still connected to 0x5614eae9cdd0 2024-04-23T21:39:22.668 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:22.667+0000 7f9212057640 -1 reset not still connected to 0x5614eaeb32b0 2024-04-23T21:39:22.668 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:22.667+0000 7f9212057640 -1 reset not still connected to 0x5614f80c1ba0 2024-04-23T21:39:22.668 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:22.667+0000 7f9212057640 -1 reset not still connected to 0x5614fae04ea0 2024-04-23T21:39:22.670 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:22.669+0000 7f9212057640 -1 reset not still connected to 0x56150723b2b0 2024-04-23T21:39:25.478 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:25.479+0000 7f9212057640 -1 reset not still connected to 0x5614e731a5b0 2024-04-23T21:39:25.482 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:25.480+0000 7f9212057640 -1 reset not still connected to 0x56150723b2b0 2024-04-23T21:39:25.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 117475328 bytes 2024-04-23T21:39:26.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:39:26.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:39:26.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:39:26.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:39:26.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:39:26.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_47 2024-04-23T21:39:26.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_47 journal 128M 2024-04-23T21:39:26.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:26.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:26.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:26.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:26.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:39:26.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:39:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:39:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_47 128M 2024-04-23T21:39:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:39:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:39:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:39:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:39:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:26.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_47 2024-04-23T21:39:26.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:39:26.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:26.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:39:26.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:39:26.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:39:26.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:39:26.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_47 2024-04-23T21:39:26.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:26.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:26.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:39:26.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:39:26.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=14 2024-04-23T21:39:26.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:39:26.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_47 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:39:26.333 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:39:26.334+0000 7fe96ab64640 -1 bdev(0x55a5abf68e00 /var/lib/ceph/osd/cluster2-2/block) aio_submit retries 1 2024-04-23T21:39:26.337 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:39:26.338+0000 7fe96eb6c640 -1 bdev(0x55a5abf68e00 /var/lib/ceph/osd/cluster2-2/block) aio_submit retries 17 2024-04-23T21:39:26.338 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:39:26.339+0000 7fe967b5e640 -1 bdev(0x55a5abf68e00 /var/lib/ceph/osd/cluster2-2/block) aio_submit retries 21 2024-04-23T21:39:28.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 23791616 bytes 2024-04-23T21:39:28.712 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:28.712+0000 7f9212057640 -1 reset not still connected to 0x5614de560000 2024-04-23T21:39:28.712 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:28.712+0000 7f9212057640 -1 reset not still connected to 0x5614e4b7a8f0 2024-04-23T21:39:28.714 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:28.714+0000 7f9212057640 -1 reset not still connected to 0x5614ebce6dd0 2024-04-23T21:39:28.715 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:28.714+0000 7f9212057640 -1 reset not still connected to 0x5614f24eb2b0 2024-04-23T21:39:28.718 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:28.717+0000 7f9212057640 -1 reset not still connected to 0x5614f603a9c0 2024-04-23T21:39:30.375 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:30.374+0000 7f9212057640 -1 reset not still connected to 0x5614de560000 2024-04-23T21:39:30.375 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:30.374+0000 7f9212057640 -1 reset not still connected to 0x5614e4b7a8f0 2024-04-23T21:39:30.375 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:30.374+0000 7f9212057640 -1 reset not still connected to 0x5614ebce6dd0 2024-04-23T21:39:30.375 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:30.374+0000 7f9212057640 -1 reset not still connected to 0x5614f24eb2b0 2024-04-23T21:39:30.375 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:30.374+0000 7f9212057640 -1 reset not still connected to 0x5614f603a9c0 2024-04-23T21:39:30.484 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:30.484+0000 7f32b7199640 -1 reset not still connected to 0x56180866a4e0 2024-04-23T21:39:30.484 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:30.484+0000 7f32b7199640 -1 reset not still connected to 0x5618087c72b0 2024-04-23T21:39:30.484 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:30.484+0000 7f32b7199640 -1 reset not still connected to 0x56181be82d00 2024-04-23T21:39:30.484 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:30.484+0000 7f32b7199640 -1 reset not still connected to 0x56181f915a00 2024-04-23T21:39:30.484 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:30.484+0000 7f32b7199640 -1 reset not still connected to 0x561827038a90 2024-04-23T21:39:31.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:39:31.208+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd85ccb40 finish: resending after timeout 2024-04-23T21:39:31.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 72934400 bytes 2024-04-23T21:39:33.973 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:33.973+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b3910ea0 2024-04-23T21:39:33.979 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:33.979+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b39904e0 2024-04-23T21:39:35.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 146362368 bytes 2024-04-23T21:39:37.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 172008448 bytes 2024-04-23T21:39:37.941 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:37.942+0000 7f32b7199640 -1 reset not still connected to 0x56181105c9c0 2024-04-23T21:39:37.941 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:37.942+0000 7f32b7199640 -1 reset not still connected to 0x561816d2b6c0 2024-04-23T21:39:37.941 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:37.942+0000 7f32b7199640 -1 reset not still connected to 0x561823989520 2024-04-23T21:39:37.942 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:37.942+0000 7f32b7199640 -1 reset not still connected to 0x5618283af380 2024-04-23T21:39:38.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 186387456 bytes 2024-04-23T21:39:39.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:600 IOs, 213467136 bytes 2024-04-23T21:39:39.899 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:39.897+0000 7f9212057640 -1 reset not still connected to 0x5614de601ad0 2024-04-23T21:39:39.901 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:39.902+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:39:40.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:39:40.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:39:40.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:39:40.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:39:40.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:39:40.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_48 2024-04-23T21:39:40.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_48 journal 128M 2024-04-23T21:39:40.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:40.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:40.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:40.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:40.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:39:40.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:40.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:39:40.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:39:40.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:40.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_48 128M 2024-04-23T21:39:40.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:40.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:40.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:40.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:40.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:39:40.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:40.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:39:40.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:39:40.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:39:40.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:40.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_48 2024-04-23T21:39:40.311 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:40.307+0000 7f9212057640 -1 reset not still connected to 0x5614de601ad0 2024-04-23T21:39:40.314 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:40.313+0000 7f9212057640 -1 reset not still connected to 0x5614f36add40 2024-04-23T21:39:40.316 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:40.316+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:39:40.316 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:40.316+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:39:40.316 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:40.316+0000 7f9212057640 -1 reset not still connected to 0x561506553790 2024-04-23T21:39:40.318 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:40.317+0000 7f9212057640 -1 reset not still connected to 0x561506a85930 2024-04-23T21:39:40.318 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:40.317+0000 7f9212057640 -1 reset not still connected to 0x56150a27f860 2024-04-23T21:39:40.318 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:40.317+0000 7f9212057640 -1 reset not still connected to 0x56150e4685b0 2024-04-23T21:39:40.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:39:40.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:40.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:39:40.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:39:40.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:39:40.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:39:40.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_48 2024-04-23T21:39:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:39:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:39:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=14 2024-04-23T21:39:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:39:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_48 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:39:42.259 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:39:42.260+0000 7fe97bfc2640 -1 reset not still connected to 0x55a5c62fc0d0 2024-04-23T21:39:42.260 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:39:42.260+0000 7fe97bfc2640 -1 reset not still connected to 0x55a5d81a76c0 2024-04-23T21:39:42.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 38057984 bytes 2024-04-23T21:39:42.990 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:39:42.988+0000 7fe97bfc2640 -1 reset not still connected to 0x55a5ddd296c0 2024-04-23T21:39:43.331 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:43.331+0000 7fc40b4f4640 -1 reset not still connected to 0x564085f0eb60 2024-04-23T21:39:43.331 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:43.331+0000 7fc40b4f4640 -1 reset not still connected to 0x56408ba7a8f0 2024-04-23T21:39:43.331 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:43.331+0000 7fc40b4f4640 -1 reset not still connected to 0x56409ff40750 2024-04-23T21:39:43.331 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:43.331+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a114be10 2024-04-23T21:39:43.331 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:43.331+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a4ed8f70 2024-04-23T21:39:43.331 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:43.331+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:39:43.334 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:43.333+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aef23380 2024-04-23T21:39:43.335 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:39:43.333+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bbf7b5f0 2024-04-23T21:39:45.505 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:39:45.505+0000 7f5874269640 -1 reset not still connected to 0x559fd80a88f0 2024-04-23T21:39:45.505 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:39:45.505+0000 7f5874269640 -1 reset not still connected to 0x559feda549c0 2024-04-23T21:39:45.505 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:39:45.505+0000 7f5874269640 -1 reset not still connected to 0x559ffccad380 2024-04-23T21:39:45.505 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:39:45.505+0000 7f5874269640 -1 reset not still connected to 0x55a009a23110 2024-04-23T21:39:46.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 91942912 bytes 2024-04-23T21:39:46.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:39:46.763+0000 7f438fb63640 -1 rbd::mirror::image_replayer::journal::Replayer: 0x55973ed86700 get_replay_status: replay not running 2024-04-23T21:39:47.717 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:39:47.718+0000 7f83ec35a640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 1 2024-04-23T21:39:47.719 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:39:47.720+0000 7f83edb5d640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 5 2024-04-23T21:39:47.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 113799168 bytes 2024-04-23T21:39:49.196 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:49.196+0000 7f32b7199640 -1 reset not still connected to 0x5618140d9110 2024-04-23T21:39:49.196 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:49.196+0000 7f32b7199640 -1 reset not still connected to 0x561818eb1c70 2024-04-23T21:39:49.197 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:39:49.198+0000 7f32b7199640 -1 reset not still connected to 0x56182d4e40d0 2024-04-23T21:39:52.269 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:52.270+0000 7f9212057640 -1 reset not still connected to 0x5614dd905040 2024-04-23T21:39:52.270 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:52.270+0000 7f9212057640 -1 reset not still connected to 0x5614dddbf2b0 2024-04-23T21:39:52.272 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:52.272+0000 7f9212057640 -1 reset not still connected to 0x5614dddbf930 2024-04-23T21:39:52.272 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:52.272+0000 7f9212057640 -1 reset not still connected to 0x5614de5295f0 2024-04-23T21:39:52.272 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:52.272+0000 7f9212057640 -1 reset not still connected to 0x5614e072a000 2024-04-23T21:39:52.272 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:52.272+0000 7f9212057640 -1 reset not still connected to 0x5614e35d1a00 2024-04-23T21:39:52.272 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:52.272+0000 7f9212057640 -1 reset not still connected to 0x5614ee548410 2024-04-23T21:39:52.272 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:52.272+0000 7f9212057640 -1 reset not still connected to 0x5614fc7ae410 2024-04-23T21:39:52.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 182009856 bytes 2024-04-23T21:39:52.839 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:52.839+0000 7f9212057640 -1 reset not still connected to 0x5614dcb0ad00 2024-04-23T21:39:52.844 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:52.843+0000 7f9212057640 -1 reset not still connected to 0x5614de601ad0 2024-04-23T21:39:52.845 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:52.845+0000 7f9212057640 -1 reset not still connected to 0x5614e5bb4b60 2024-04-23T21:39:52.845 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:52.845+0000 7f9212057640 -1 reset not still connected to 0x5614f36add40 2024-04-23T21:39:52.845 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:52.845+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:39:52.845 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:52.845+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:39:52.845 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:52.845+0000 7f9212057640 -1 reset not still connected to 0x561506553790 2024-04-23T21:39:52.846 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:52.845+0000 7f9212057640 -1 reset not still connected to 0x561506a85930 2024-04-23T21:39:53.477 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:53.477+0000 7f9212057640 -1 reset not still connected to 0x5614e5bb4b60 2024-04-23T21:39:53.477 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:53.477+0000 7f9212057640 -1 reset not still connected to 0x5614f36add40 2024-04-23T21:39:53.477 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:53.477+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:39:53.478 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:53.477+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:39:53.478 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:53.477+0000 7f9212057640 -1 reset not still connected to 0x561506553790 2024-04-23T21:39:53.480 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:53.478+0000 7f9212057640 -1 reset not still connected to 0x561506a85930 2024-04-23T21:39:53.480 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:53.478+0000 7f9212057640 -1 reset not still connected to 0x56150a27f860 2024-04-23T21:39:53.480 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:53.480+0000 7f9212057640 -1 reset not still connected to 0x56150e4685b0 2024-04-23T21:39:54.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:39:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:39:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:39:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:39:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:39:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_49 2024-04-23T21:39:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_49 journal 128M 2024-04-23T21:39:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:39:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:39:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:39:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_49 128M 2024-04-23T21:39:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:39:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:39:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:39:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:39:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:39:54.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_49 2024-04-23T21:39:54.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:39:54.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:54.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:39:54.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:39:54.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:39:54.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:39:54.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_49 2024-04-23T21:39:54.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:39:54.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:39:54.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:39:54.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:39:54.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=11 2024-04-23T21:39:54.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:39:54.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_49 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:39:55.295 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:55.294+0000 7f9212057640 -1 reset not still connected to 0x5614f36add40 2024-04-23T21:39:55.303 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:55.298+0000 7f9212057640 -1 reset not still connected to 0x5614ffd2f040 2024-04-23T21:39:55.303 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:55.298+0000 7f9212057640 -1 reset not still connected to 0x561505fbe820 2024-04-23T21:39:55.303 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:55.298+0000 7f9212057640 -1 reset not still connected to 0x561506553790 2024-04-23T21:39:55.303 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:55.298+0000 7f9212057640 -1 reset not still connected to 0x561506a85930 2024-04-23T21:39:55.303 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:55.304+0000 7f9212057640 -1 reset not still connected to 0x56150e4685b0 2024-04-23T21:39:58.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 67303424 bytes 2024-04-23T21:39:58.195 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:58.192+0000 7f9212057640 -1 reset not still connected to 0x5614ebce6dd0 2024-04-23T21:39:58.196 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:58.195+0000 7f9212057640 -1 reset not still connected to 0x5614f603a9c0 2024-04-23T21:39:58.196 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:39:58.197+0000 7f9212057640 -1 reset not still connected to 0x56150723a410 2024-04-23T21:40:03.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 152421376 bytes 2024-04-23T21:40:04.687 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:04.686+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac2d4dd0 2024-04-23T21:40:04.688 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:04.689+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b3910ea0 2024-04-23T21:40:04.688 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:04.689+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b39904e0 2024-04-23T21:40:04.692 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:04.692+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b8b271e0 2024-04-23T21:40:05.281 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:05.282+0000 7fc40b4f4640 -1 reset not still connected to 0x56409a8809c0 2024-04-23T21:40:05.282 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:05.282+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bd19a750 2024-04-23T21:40:05.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:40:05.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:40:05.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:40:05.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:40:05.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:40:05.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_50 2024-04-23T21:40:05.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image_and_enable_mirror cluster2 mirror image_50 journal 128M 2024-04-23T21:40:05.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:40:05.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:40:05.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:40:05.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:40:05.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:40:05.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:40:05.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:40:05.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n journal ']' 2024-04-23T21:40:05.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:40:05.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_50 128M 2024-04-23T21:40:05.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:40:05.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:40:05.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:40:05.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:40:05.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:40:05.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:40:05.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:40:05.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 128M ']' 2024-04-23T21:40:05.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=128M 2024-04-23T21:40:05.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:40:05.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock,journaling mirror/image_50 2024-04-23T21:40:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ is_pool_mirror_mode_image mirror 2024-04-23T21:40:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:40:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' pool = image ']' 2024-04-23T21:40:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ case "${pool}" in 2024-04-23T21:40:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 1 2024-04-23T21:40:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n '' ']' 2024-04-23T21:40:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_50 2024-04-23T21:40:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:40:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:40:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:40:05.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:40:05.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=7 2024-04-23T21:40:05.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:40:05.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_50 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:40:06.011 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:06.011+0000 7f9212057640 -1 reset not still connected to 0x5614e35d1a00 2024-04-23T21:40:06.011 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:06.011+0000 7f9212057640 -1 reset not still connected to 0x5614ee548410 2024-04-23T21:40:06.012 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:06.011+0000 7f9212057640 -1 reset not still connected to 0x5614fc7ae410 2024-04-23T21:40:06.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 13332480 bytes 2024-04-23T21:40:06.874 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8023210661304949) > weight (0.25) 2024-04-23T21:40:06.874 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9763703872121384) > weight (0.25) 2024-04-23T21:40:06.874 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4303742226658187) > weight (0.25) 2024-04-23T21:40:06.874 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2024-04-23T21:40:06.875 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2024-04-23T21:40:06.875 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2024-04-23T21:40:06.877 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:40:06.876+0000 7faf29ed7640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 61148) UID: 1000 2024-04-23T21:40:07.786 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:40:07.785+0000 7f32b7199640 -1 reset not still connected to 0x56181119c680 2024-04-23T21:40:07.786 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:40:07.787+0000 7f32b7199640 -1 reset not still connected to 0x5618140d9110 2024-04-23T21:40:07.787 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:40:07.787+0000 7f32b7199640 -1 reset not still connected to 0x561818eb1c70 2024-04-23T21:40:07.787 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:40:07.787+0000 7f32b7199640 -1 reset not still connected to 0x56182d4e40d0 2024-04-23T21:40:08.542 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:08.542+0000 7f9212057640 -1 reset not still connected to 0x5614dd1d5380 2024-04-23T21:40:08.546 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:08.544+0000 7f9212057640 -1 reset not still connected to 0x5614ddd505b0 2024-04-23T21:40:08.547 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:08.547+0000 7f9212057640 -1 reset not still connected to 0x5614de3b0b60 2024-04-23T21:40:08.547 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:08.547+0000 7f9212057640 -1 reset not still connected to 0x5614de3b1380 2024-04-23T21:40:08.548 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:08.547+0000 7f9212057640 -1 reset not still connected to 0x5614de560000 2024-04-23T21:40:08.548 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:08.547+0000 7f9212057640 -1 reset not still connected to 0x5614e4b7a8f0 2024-04-23T21:40:08.548 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:08.547+0000 7f9212057640 -1 reset not still connected to 0x5614ebce6dd0 2024-04-23T21:40:08.548 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:08.547+0000 7f9212057640 -1 reset not still connected to 0x5614f24eb2b0 2024-04-23T21:40:08.548 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:08.547+0000 7f9212057640 -1 reset not still connected to 0x5614f603a9c0 2024-04-23T21:40:10.102 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:40:10.102+0000 7f32b7199640 -1 reset not still connected to 0x56182306fe10 2024-04-23T21:40:10.102 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:40:10.102+0000 7f32b7199640 -1 reset not still connected to 0x5618246fb790 2024-04-23T21:40:10.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 62303232 bytes 2024-04-23T21:40:10.875 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2024-04-23T21:40:10.876 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:40:10.876 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2024-04-23T21:40:10.876 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2024-04-23T21:40:10.876 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2024-04-23T21:40:10.879 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2024-04-23T21:40:10.879 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2024-04-23T21:40:11.265 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:40:11.265+0000 7f32b7199640 -1 reset not still connected to 0x5618109795f0 2024-04-23T21:40:11.265 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:40:11.265+0000 7f32b7199640 -1 reset not still connected to 0x56181105c9c0 2024-04-23T21:40:11.265 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:40:11.265+0000 7f32b7199640 -1 reset not still connected to 0x561816d2b6c0 2024-04-23T21:40:11.266 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:40:11.265+0000 7f32b7199640 -1 reset not still connected to 0x56182326f860 2024-04-23T21:40:11.266 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:40:11.265+0000 7f32b7199640 -1 reset not still connected to 0x561823989520 2024-04-23T21:40:11.266 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:40:11.265+0000 7f32b7199640 -1 reset not still connected to 0x5618275f4750 2024-04-23T21:40:11.266 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:40:11.265+0000 7f32b7199640 -1 reset not still connected to 0x5618283af380 2024-04-23T21:40:12.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:40:12.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:40:12.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:40:12.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:40:12.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_pool_images cluster2 mirror 50 2024-04-23T21:40:12.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:40:12.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:40:12.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_count=50 2024-04-23T21:40:12.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:40:12.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local count 2024-04-23T21:40:12.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_count=0 2024-04-23T21:40:12.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:40:12.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 1 40 2024-04-23T21:40:12.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in `seq 1 40` 2024-04-23T21:40:12.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 1 -ne 1 2024-04-23T21:40:12.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 -p mirror mirror pool status 2024-04-23T21:40:12.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep 'images: ' 2024-04-23T21:40:12.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut '-d ' -f 2 2024-04-23T21:40:12.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: failed to query services: (13) Permission denied 2024-04-23T21:40:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ count=50 2024-04-23T21:40:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 50 = 50 2024-04-23T21:40:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:40:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_pool_healthy cluster2 mirror 2024-04-23T21:40:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:40:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:40:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:40:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state 2024-04-23T21:40:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 1 40 2024-04-23T21:40:12.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in `seq 1 40` 2024-04-23T21:40:12.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 1 -ne 1 2024-04-23T21:40:12.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 -p mirror mirror pool status 2024-04-23T21:40:12.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep 'image health:' 2024-04-23T21:40:12.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut '-d ' -f 3 2024-04-23T21:40:12.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: failed to query services: (13) Permission denied 2024-04-23T21:40:12.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ state=WARNING 2024-04-23T21:40:12.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test WARNING = ERROR 2024-04-23T21:40:12.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test WARNING = OK 2024-04-23T21:40:12.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in `seq 1 40` 2024-04-23T21:40:12.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 -ne 1 2024-04-23T21:40:12.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 30 2024-04-23T21:40:12.667 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:12.667+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a46511e0 2024-04-23T21:40:12.667 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:12.667+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a97b6a90 2024-04-23T21:40:12.668 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:12.669+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aa6d69c0 2024-04-23T21:40:12.670 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:12.670+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aebc85b0 2024-04-23T21:40:12.670 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:12.670+0000 7fc40b4f4640 -1 reset not still connected to 0x5640af45e000 2024-04-23T21:40:12.670 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:12.670+0000 7fc40b4f4640 -1 reset not still connected to 0x5640afc1cdd0 2024-04-23T21:40:12.670 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:12.670+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4eedad0 2024-04-23T21:40:12.670 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:12.670+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bd19a750 2024-04-23T21:40:18.805 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:18.806+0000 7f9212057640 -1 reset not still connected to 0x5614de3b0b60 2024-04-23T21:40:18.809 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:18.810+0000 7f9212057640 -1 reset not still connected to 0x5614de3b1380 2024-04-23T21:40:18.812 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:18.812+0000 7f9212057640 -1 reset not still connected to 0x5614de560000 2024-04-23T21:40:18.813 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:18.812+0000 7f9212057640 -1 reset not still connected to 0x5614e4b7a8f0 2024-04-23T21:40:18.813 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:18.812+0000 7f9212057640 -1 reset not still connected to 0x5614ebce6dd0 2024-04-23T21:40:18.813 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:18.812+0000 7f9212057640 -1 reset not still connected to 0x5614f24eb2b0 2024-04-23T21:40:18.813 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:18.812+0000 7f9212057640 -1 reset not still connected to 0x5614f603a9c0 2024-04-23T21:40:18.813 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:18.812+0000 7f9212057640 -1 reset not still connected to 0x56150723a410 2024-04-23T21:40:19.001 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:19.000+0000 7f9212057640 -1 reset not still connected to 0x5614ddd505b0 2024-04-23T21:40:19.001 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:19.000+0000 7f9212057640 -1 reset not still connected to 0x5614de3b0b60 2024-04-23T21:40:19.001 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:19.000+0000 7f9212057640 -1 reset not still connected to 0x5614de3b1380 2024-04-23T21:40:19.001 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:19.001+0000 7f9212057640 -1 reset not still connected to 0x5614de560000 2024-04-23T21:40:19.001 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:19.001+0000 7f9212057640 -1 reset not still connected to 0x5614e4b7a8f0 2024-04-23T21:40:19.001 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:19.001+0000 7f9212057640 -1 reset not still connected to 0x5614ebce6dd0 2024-04-23T21:40:19.001 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:19.001+0000 7f9212057640 -1 reset not still connected to 0x5614f24eb2b0 2024-04-23T21:40:19.001 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:19.001+0000 7f9212057640 -1 reset not still connected to 0x5614f603a9c0 2024-04-23T21:40:19.002 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:19.001+0000 7f9212057640 -1 reset not still connected to 0x56150723a410 2024-04-23T21:40:19.759 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:19.758+0000 7f9212057640 -1 reset not still connected to 0x5614d298a9c0 2024-04-23T21:40:19.760 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:19.761+0000 7f9212057640 -1 reset not still connected to 0x5614d298b380 2024-04-23T21:40:19.760 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:19.761+0000 7f9212057640 -1 reset not still connected to 0x5614dbf71790 2024-04-23T21:40:19.767 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:19.765+0000 7f9212057640 -1 reset not still connected to 0x5614eaeb32b0 2024-04-23T21:40:19.767 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:19.765+0000 7f9212057640 -1 reset not still connected to 0x5614f80c1ba0 2024-04-23T21:40:19.768 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:19.769+0000 7f9212057640 -1 reset not still connected to 0x5614fae04ea0 2024-04-23T21:40:19.768 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:19.769+0000 7f9212057640 -1 reset not still connected to 0x56150723b2b0 2024-04-23T21:40:20.254 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:20.254+0000 7f9212057640 -1 reset not still connected to 0x5614dddbf930 2024-04-23T21:40:20.254 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:20.254+0000 7f9212057640 -1 reset not still connected to 0x5614de5295f0 2024-04-23T21:40:20.254 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:20.254+0000 7f9212057640 -1 reset not still connected to 0x5614e072a000 2024-04-23T21:40:20.255 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:20.254+0000 7f9212057640 -1 reset not still connected to 0x5614e35d1a00 2024-04-23T21:40:20.255 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:20.254+0000 7f9212057640 -1 reset not still connected to 0x5614ee548410 2024-04-23T21:40:20.255 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:20.254+0000 7f9212057640 -1 reset not still connected to 0x5614fc7ae410 2024-04-23T21:40:20.279 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2024-04-23T21:40:20.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2024-04-23T21:40:20.280 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2024-04-23T21:40:20.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:40:20.281+0000 7f1474850640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 51527) UID: 1000 2024-04-23T21:40:20.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:40:20.956+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851da40 finish: resending after timeout 2024-04-23T21:40:21.712 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:21.708+0000 7f9212057640 -1 reset not still connected to 0x5614d298a9c0 2024-04-23T21:40:21.712 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:21.708+0000 7f9212057640 -1 reset not still connected to 0x5614d298b380 2024-04-23T21:40:21.712 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:21.712+0000 7f9212057640 -1 reset not still connected to 0x5614dbf71790 2024-04-23T21:40:21.712 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:21.712+0000 7f9212057640 -1 reset not still connected to 0x5614e731a5b0 2024-04-23T21:40:21.712 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:21.712+0000 7f9212057640 -1 reset not still connected to 0x5614eae9cdd0 2024-04-23T21:40:21.712 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:21.712+0000 7f9212057640 -1 reset not still connected to 0x5614eaeb32b0 2024-04-23T21:40:21.712 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:21.712+0000 7f9212057640 -1 reset not still connected to 0x5614f80c1ba0 2024-04-23T21:40:21.712 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:21.712+0000 7f9212057640 -1 reset not still connected to 0x5614fae04ea0 2024-04-23T21:40:21.712 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:21.712+0000 7f9212057640 -1 reset not still connected to 0x56150723b2b0 2024-04-23T21:40:25.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:40:25.956+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851da40 finish: resending after timeout 2024-04-23T21:40:28.280 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2024-04-23T21:40:28.281 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:40:28.281 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2024-04-23T21:40:28.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2024-04-23T21:40:28.281 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2024-04-23T21:40:28.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2024-04-23T21:40:28.284 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2024-04-23T21:40:30.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:40:30.957+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851da40 finish: resending after timeout 2024-04-23T21:40:35.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:40:35.958+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851da40 finish: resending after timeout 2024-04-23T21:40:40.281 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:40.282+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b28c95f0 2024-04-23T21:40:40.282 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:40.282+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:40:40.282 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:40.282+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:40:40.369 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:40.369+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aebc85b0 2024-04-23T21:40:40.369 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:40.369+0000 7fc40b4f4640 -1 reset not still connected to 0x5640af45e000 2024-04-23T21:40:40.369 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:40.369+0000 7fc40b4f4640 -1 reset not still connected to 0x5640afc1cdd0 2024-04-23T21:40:40.369 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:40.369+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4eedad0 2024-04-23T21:40:40.369 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:40.369+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bd19a750 2024-04-23T21:40:40.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:40:40.958+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851da40 finish: resending after timeout 2024-04-23T21:40:42.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 -p mirror mirror pool status 2024-04-23T21:40:42.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep 'image health:' 2024-04-23T21:40:42.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut '-d ' -f 3 2024-04-23T21:40:42.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: failed to query services: (13) Permission denied 2024-04-23T21:40:42.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ state=WARNING 2024-04-23T21:40:42.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test WARNING = ERROR 2024-04-23T21:40:42.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test WARNING = OK 2024-04-23T21:40:42.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in `seq 1 40` 2024-04-23T21:40:42.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 -ne 1 2024-04-23T21:40:42.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 30 2024-04-23T21:40:44.219 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:44.219+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b1d38270 2024-04-23T21:40:44.219 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:44.219+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b28c8820 2024-04-23T21:40:44.219 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:44.219+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b28c95f0 2024-04-23T21:40:44.219 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:44.219+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:40:44.220 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:44.219+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:40:45.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:40:45.959+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851da40 finish: resending after timeout 2024-04-23T21:40:46.510 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:46.509+0000 7f9212057640 -1 reset not still connected to 0x5614e731a5b0 2024-04-23T21:40:46.510 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:46.511+0000 7f9212057640 -1 reset not still connected to 0x5614eaeb32b0 2024-04-23T21:40:46.511 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:46.511+0000 7f9212057640 -1 reset not still connected to 0x5614f80c1ba0 2024-04-23T21:40:46.514 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:46.514+0000 7f9212057640 -1 reset not still connected to 0x5614fae04ea0 2024-04-23T21:40:46.514 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:46.515+0000 7f9212057640 -1 reset not still connected to 0x56150723b2b0 2024-04-23T21:40:47.950 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:47.950+0000 7f9212057640 -1 reset not still connected to 0x5614eaeb32b0 2024-04-23T21:40:47.950 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:47.950+0000 7f9212057640 -1 reset not still connected to 0x5614f80c1ba0 2024-04-23T21:40:47.950 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:47.950+0000 7f9212057640 -1 reset not still connected to 0x5614fae04ea0 2024-04-23T21:40:47.950 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:47.950+0000 7f9212057640 -1 reset not still connected to 0x56150723b2b0 2024-04-23T21:40:48.001 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:48.001+0000 7f9212057640 -1 reset not still connected to 0x5614d298a9c0 2024-04-23T21:40:48.001 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:48.001+0000 7f9212057640 -1 reset not still connected to 0x5614d298b380 2024-04-23T21:40:48.001 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:48.001+0000 7f9212057640 -1 reset not still connected to 0x5614dbf71790 2024-04-23T21:40:48.001 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:48.001+0000 7f9212057640 -1 reset not still connected to 0x5614e731a5b0 2024-04-23T21:40:48.004 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:48.004+0000 7f9212057640 -1 reset not still connected to 0x5614eaeb32b0 2024-04-23T21:40:48.004 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:48.004+0000 7f9212057640 -1 reset not still connected to 0x5614f80c1ba0 2024-04-23T21:40:48.004 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:48.004+0000 7f9212057640 -1 reset not still connected to 0x5614fae04ea0 2024-04-23T21:40:48.004 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:40:48.004+0000 7f9212057640 -1 reset not still connected to 0x56150723b2b0 2024-04-23T21:40:48.500 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:48.501+0000 7fc40b4f4640 -1 reset not still connected to 0x56409a8809c0 2024-04-23T21:40:48.503 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:48.503+0000 7fc40b4f4640 -1 reset not still connected to 0x56409c601ba0 2024-04-23T21:40:48.503 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:48.503+0000 7fc40b4f4640 -1 reset not still connected to 0x56409c62c820 2024-04-23T21:40:48.503 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:48.503+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a46511e0 2024-04-23T21:40:48.503 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:48.503+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a97b6a90 2024-04-23T21:40:48.503 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:48.503+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aa6d69c0 2024-04-23T21:40:48.505 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:48.505+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aebc85b0 2024-04-23T21:40:48.505 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:48.505+0000 7fc40b4f4640 -1 reset not still connected to 0x5640af45e000 2024-04-23T21:40:48.505 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:48.505+0000 7fc40b4f4640 -1 reset not still connected to 0x5640afc1cdd0 2024-04-23T21:40:48.505 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:48.505+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4eedad0 2024-04-23T21:40:48.505 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:48.505+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bd19a750 2024-04-23T21:40:50.413 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:50.410+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a9049c70 2024-04-23T21:40:50.415 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:50.415+0000 7fc40b4f4640 -1 reset not still connected to 0x5640acc968f0 2024-04-23T21:40:50.415 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:50.415+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ad4ce340 2024-04-23T21:40:50.415 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:50.415+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b1d38270 2024-04-23T21:40:50.416 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:50.415+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b28c8820 2024-04-23T21:40:50.416 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:50.415+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b28c95f0 2024-04-23T21:40:50.416 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:50.415+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b4f93520 2024-04-23T21:40:50.416 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:50.415+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c0cf2340 2024-04-23T21:40:50.416 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:40:50.416+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c14ac1a0 2024-04-23T21:40:50.661 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:40:50.660+0000 7f5864e0f640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 1 2024-04-23T21:40:50.663 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:40:50.663+0000 7f5860606640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 8 2024-04-23T21:40:50.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:40:50.958+0000 7f382edc3640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851da40 finish: resending after timeout 2024-04-23T21:40:51.383 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:40:51.383+0000 7fe96eb6c640 -1 bdev(0x55a5abf68e00 /var/lib/ceph/osd/cluster2-2/block) aio_submit retries 9 2024-04-23T21:40:51.384 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:40:51.384+0000 7fe96e36b640 -1 bdev(0x55a5abf68e00 /var/lib/ceph/osd/cluster2-2/block) aio_submit retries 9 2024-04-23T21:40:55.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:40:55.959+0000 7f382e5c2640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x55cbd851da40 finish: resending after timeout 2024-04-23T21:40:57.392 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:40:57.392+0000 7f83ec35a640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 6 2024-04-23T21:40:57.394 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:40:57.394+0000 7f83e9354640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 6 2024-04-23T21:41:00.364 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:41:00.365+0000 7f32b7199640 -1 reset not still connected to 0x561823989520 2024-04-23T21:41:00.368 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:41:00.367+0000 7f32b7199640 -1 reset not still connected to 0x5618283af380 2024-04-23T21:41:00.368 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:41:00.367+0000 7f32b7199640 -1 reset not still connected to 0x561829eaa0d0 2024-04-23T21:41:06.486 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:41:06.480+0000 7fc40b4f4640 -1 reset not still connected to 0x56408b23b520 2024-04-23T21:41:06.487 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:41:06.481+0000 7fc40b4f4640 -1 reset not still connected to 0x56408ba7a8f0 2024-04-23T21:41:06.487 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:41:06.481+0000 7fc40b4f4640 -1 reset not still connected to 0x56409ff40750 2024-04-23T21:41:06.487 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:41:06.485+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aa78ab60 2024-04-23T21:41:06.487 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:41:06.485+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ac7431e0 2024-04-23T21:41:06.487 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:41:06.487+0000 7fc40b4f4640 -1 reset not still connected to 0x5640ad8ecea0 2024-04-23T21:41:06.487 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:41:06.487+0000 7fc40b4f4640 -1 reset not still connected to 0x5640aef23380 2024-04-23T21:41:06.487 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:41:06.487+0000 7fc40b4f4640 -1 reset not still connected to 0x5640afc6fd40 2024-04-23T21:41:12.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 -p mirror mirror pool status 2024-04-23T21:41:12.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep 'image health:' 2024-04-23T21:41:12.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut '-d ' -f 3 2024-04-23T21:41:12.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: failed to query services: (13) Permission denied 2024-04-23T21:41:12.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ state=OK 2024-04-23T21:41:12.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test OK = ERROR 2024-04-23T21:41:12.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test OK = OK 2024-04-23T21:41:12.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:12.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_pool_images cluster1 mirror 50 2024-04-23T21:41:12.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:12.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:12.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_count=50 2024-04-23T21:41:12.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:41:12.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local count 2024-04-23T21:41:12.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_count=0 2024-04-23T21:41:12.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:41:12.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 1 40 2024-04-23T21:41:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in `seq 1 40` 2024-04-23T21:41:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 1 -ne 1 2024-04-23T21:41:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 -p mirror mirror pool status 2024-04-23T21:41:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep 'images: ' 2024-04-23T21:41:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut '-d ' -f 2 2024-04-23T21:41:12.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: failed to query services: (13) Permission denied 2024-04-23T21:41:12.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ count=50 2024-04-23T21:41:12.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 50 = 50 2024-04-23T21:41:12.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:12.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_pool_healthy cluster1 mirror 2024-04-23T21:41:12.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:12.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:12.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:41:12.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state 2024-04-23T21:41:12.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 1 40 2024-04-23T21:41:12.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in `seq 1 40` 2024-04-23T21:41:12.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 1 -ne 1 2024-04-23T21:41:12.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 -p mirror mirror pool status 2024-04-23T21:41:12.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep 'image health:' 2024-04-23T21:41:12.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut '-d ' -f 3 2024-04-23T21:41:12.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: failed to query services: (13) Permission denied 2024-04-23T21:41:12.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ state=OK 2024-04-23T21:41:12.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test OK = ERROR 2024-04-23T21:41:12.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test OK = OK 2024-04-23T21:41:12.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:12.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ testlog 'TEST: compare many images' 2024-04-23T21:41:12.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ tee -a /tmp/tmp.8Tyc9aSFBO/rbd-mirror.test.log 2024-04-23T21:41:12.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ date '+%F %T' 2024-04-23T21:41:12.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo 2024-04-23 21:41:12 TEST: compare many images 2024-04-23T21:41:12.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:2024-04-23 21:41:12 TEST: compare many images 2024-04-23T21:41:12.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 1 50 2024-04-23T21:41:12.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:41:12.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_1 2024-04-23T21:41:12.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_1 snap 2024-04-23T21:41:12.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:41:12.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:12.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:41:12.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:41:12.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_1@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:41:13.349 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:41:13.350+0000 7f83ed35c640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 11 2024-04-23T21:41:13.349 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:41:13.351+0000 7f83e9b55640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 12 2024-04-23T21:41:13.350 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:41:13.351+0000 7f83e8b53640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 15 2024-04-23T21:41:15.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_1 2024-04-23T21:41:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:41:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_1 started 2024-04-23T21:41:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:41:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:41:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:41:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:15.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:41:16.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_1 started 2024-04-23T21:41:16.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:16.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:16.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:41:16.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:41:16.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:41:16.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:41:16.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_1 2024-04-23T21:41:16.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:41:16.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:41:16.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:16.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:16.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:41:16.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:41:16.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:16.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:41:16.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:16.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:41:16.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:16.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_1 2024-04-23T21:41:16.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:41:16.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:16.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:16.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:16.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:41:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:16.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:41:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:41:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:41:16.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:16.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:16.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:16.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:16.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_1 2024-04-23T21:41:16.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 5 closest matches: 2024-04-23T21:41:16.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_13 2024-04-23T21:41:16.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_15 2024-04-23T21:41:16.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_17 2024-04-23T21:41:16.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_18 2024-04-23T21:41:16.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_19 2024-04-23T21:41:16.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_1 2024-04-23T21:41:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:41:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:16.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:41:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:41:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:16.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:16.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:41:16.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:41:16.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:16.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:16.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:16.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:16.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_1 2024-04-23T21:41:17.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 3 closest matches: 2024-04-23T21:41:17.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_11 2024-04-23T21:41:17.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_12 2024-04-23T21:41:17.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_16 2024-04-23T21:41:17.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:17.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:17.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_1 2024-04-23T21:41:17.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:41:17.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:17.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:17.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:17.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:41:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:17.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:41:17.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:17.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:17.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:17.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:17.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:17.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:17.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:17.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:41:17.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:41:17.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:17.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:17.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:17.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:17.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_1 2024-04-23T21:41:17.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 1 closest matches: 2024-04-23T21:41:17.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_14 2024-04-23T21:41:17.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:17.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:17.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_1 2024-04-23T21:41:17.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:41:17.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:17.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:17.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:41:17.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:41:17.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:17.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:17.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:17.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:17.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:17.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:17.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:17.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:41:17.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:41:17.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:41:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:41:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:41:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:41:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:41:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:41:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:41:17.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:41:17.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:17.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:17.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:17.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_1 2024-04-23T21:41:17.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:41:17.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:41:17.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:41:17.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:41:17.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:41:17.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:41:17.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:41:17.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:17.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_1 2024-04-23T21:41:17.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:41:17.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:41:17.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:17.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:41:17.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:41:17.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_1 2024-04-23T21:41:17.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:41:17.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:41:17.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:17.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:41:17.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:41:17.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:41:17.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:41:17.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:17.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:41:17.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_1 2024-04-23T21:41:17.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:17.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:17.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:41:17.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:17.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_1 ']' 2024-04-23T21:41:17.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_1' 2024-04-23T21:41:17.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_1 2024-04-23T21:41:17.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:17.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:17.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:17.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:17.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:17.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:17.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:17.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:17.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:17.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:17.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_1 2024-04-23T21:41:17.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:17.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:17.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:17.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:17.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:17.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:17.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:17.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:17.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:17.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:17.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:17.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:17.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:17.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:17.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:17.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:17.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:17.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:17.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_1 2024-04-23T21:41:17.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 5 closest matches: 2024-04-23T21:41:17.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_13 2024-04-23T21:41:17.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_15 2024-04-23T21:41:17.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_17 2024-04-23T21:41:17.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_18 2024-04-23T21:41:17.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_19 2024-04-23T21:41:17.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:17.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:17.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_1 2024-04-23T21:41:17.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:17.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:17.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:17.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:17.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:17.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:17.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:17.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:17.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:17.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:17.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:17.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:17.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:17.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:17.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:17.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:17.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:17.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:17.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:17.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:17.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:17.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:17.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:17.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:17.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:17.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:17.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:17.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:17.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:17.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:17.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:17.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:17.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:17.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_1 2024-04-23T21:41:17.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 3 closest matches: 2024-04-23T21:41:17.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_11 2024-04-23T21:41:17.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_12 2024-04-23T21:41:17.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_16 2024-04-23T21:41:17.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:17.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:17.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_1 2024-04-23T21:41:17.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:41:17.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:17.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:17.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:17.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:41:17.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:17.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:17.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:17.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:41:17.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:17.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:17.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:17.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:41:17.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:17.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:17.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:41:17.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:17.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:17.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:17.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:17.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:17.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:17.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:17.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:17.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:17.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:17.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:17.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:17.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:17.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:17.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:17.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:17.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:17.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_1 2024-04-23T21:41:17.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 1 closest matches: 2024-04-23T21:41:17.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_14 2024-04-23T21:41:17.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:17.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:17.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_1 2024-04-23T21:41:17.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:41:17.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:17.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:17.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:41:17.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:41:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:41:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:41:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:41:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:41:17.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:17.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:17.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:41:17.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:41:17.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:17.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:17.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:17.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:17.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:17.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:17.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:17.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:41:17.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:41:17.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:41:17.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:17.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:17.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:17.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_1 2024-04-23T21:41:17.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:17.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_1 2024-04-23T21:41:17.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:17.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:17.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_1 2024-04-23T21:41:17.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_1 '' 2024-04-23T21:41:17.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:17.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:17.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_1 2024-04-23T21:41:17.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:17.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_1.status 2024-04-23T21:41:17.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_1.status 2024-04-23T21:41:17.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:17.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.status 2024-04-23T21:41:17.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_1 2024-04-23T21:41:17.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.status 2024-04-23T21:41:17.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:17.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:17.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:17.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=9115], [object_number=10, tag_tid=1, entry_tid=9114]]], state=connected] 2024-04-23T21:41:17.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=8, tag_tid=1, entry_tid=9116], [object_number=11, tag_tid=1, entry_tid=9115], [object_number=10, tag_tid=1, entry_tid=9114], [object_number=9, tag_tid=1, entry_tid=9113]]], state=connected] 2024-04-23T21:41:17.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.status 2024-04-23T21:41:17.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:17.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_1 2024-04-23T21:41:17.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:17.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:17.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_1 2024-04-23T21:41:17.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_1 '..*' 2024-04-23T21:41:17.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:17.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:17.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_1 2024-04-23T21:41:17.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:17.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_1.status 2024-04-23T21:41:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_1.status 2024-04-23T21:41:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.status 2024-04-23T21:41:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_1 2024-04-23T21:41:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.status 2024-04-23T21:41:17.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:17.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:17.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:17.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=9115], [object_number=10, tag_tid=1, entry_tid=9114]]], state=connected] 2024-04-23T21:41:17.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=8, tag_tid=1, entry_tid=9116], [object_number=11, tag_tid=1, entry_tid=9115], [object_number=10, tag_tid=1, entry_tid=9114], [object_number=9, tag_tid=1, entry_tid=9113]]], state=connected] 2024-04-23T21:41:17.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.status 2024-04-23T21:41:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=8, tag_tid=1, entry_tid=9116' 2024-04-23T21:41:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=8, tag_tid=1, entry_tid=9116' 2024-04-23T21:41:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=8, tag_tid=1, entry_tid=9116' '!=' '' 2024-04-23T21:41:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:41:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=8, tag_tid=1, entry_tid=9116' = '' 2024-04-23T21:41:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=8, tag_tid=1, entry_tid=9116' 2024-04-23T21:41:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:41:17.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:17.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:41:17.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=8, tag_tid=1, entry_tid=9116' 2024-04-23T21:41:17.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:41:17.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:17.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:41:17.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:17.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:41:17.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:18.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:41:18.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=8, tag_tid=1, entry_tid=9116' 2024-04-23T21:41:18.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:41:18.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:18.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=9116 2024-04-23T21:41:18.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 9116 2024-04-23T21:41:18.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:41:18.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:18.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:41:18.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_1 2024-04-23T21:41:18.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:18.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:18.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:41:18.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:18.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_1 ']' 2024-04-23T21:41:18.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_1' 2024-04-23T21:41:18.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_1 2024-04-23T21:41:18.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:18.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:18.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:18.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:18.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:18.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:18.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:18.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:18.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:18.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:18.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_1 2024-04-23T21:41:18.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:18.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:18.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:18.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:18.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:18.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:18.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:18.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:18.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:18.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:18.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:18.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:18.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:18.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:18.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:18.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:18.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:18.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:18.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:18.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:18.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:18.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:18.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:18.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:18.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:18.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:18.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:18.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:18.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:18.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:18.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:18.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:18.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:18.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_1 2024-04-23T21:41:18.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 5 closest matches: 2024-04-23T21:41:18.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_13 2024-04-23T21:41:18.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_15 2024-04-23T21:41:18.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_17 2024-04-23T21:41:18.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_18 2024-04-23T21:41:18.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_19 2024-04-23T21:41:18.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:18.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:18.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_1 2024-04-23T21:41:18.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:18.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:18.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:18.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:18.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:18.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:18.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:18.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:18.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:18.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:18.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:18.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:18.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:18.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:18.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:18.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:18.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:18.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:18.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:18.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:18.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:18.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:18.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:18.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:18.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:18.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:18.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:18.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:18.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:18.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:18.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:18.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:18.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:18.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_1 2024-04-23T21:41:18.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 3 closest matches: 2024-04-23T21:41:18.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_11 2024-04-23T21:41:18.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_12 2024-04-23T21:41:18.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_16 2024-04-23T21:41:18.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:18.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:18.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_1 2024-04-23T21:41:18.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:41:18.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:18.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:18.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:18.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:41:18.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:18.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:18.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:18.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:41:18.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:41:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:41:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:18.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:18.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:18.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:18.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:18.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:18.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:18.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:18.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:18.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:18.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_1 2024-04-23T21:41:18.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 1 closest matches: 2024-04-23T21:41:18.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_14 2024-04-23T21:41:18.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:18.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:18.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_1 2024-04-23T21:41:18.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:41:18.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:18.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:18.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:41:18.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:41:18.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:18.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:18.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:18.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:41:18.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:18.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:18.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:18.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:41:18.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:18.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:41:18.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:41:18.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:18.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:18.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:41:18.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:41:18.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:18.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:18.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:18.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:18.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:18.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:18.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:18.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:41:18.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:41:18.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:41:18.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:18.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:18.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:18.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_1 2024-04-23T21:41:18.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:18.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_1 2024-04-23T21:41:18.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:18.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:18.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_1 2024-04-23T21:41:18.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_1 '' 2024-04-23T21:41:18.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:18.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:18.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_1 2024-04-23T21:41:18.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:18.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_1.status 2024-04-23T21:41:18.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_1.status 2024-04-23T21:41:18.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:18.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.status 2024-04-23T21:41:18.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_1 2024-04-23T21:41:18.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.status 2024-04-23T21:41:18.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:18.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:18.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:18.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=9115], [object_number=10, tag_tid=1, entry_tid=9114]]], state=connected] 2024-04-23T21:41:18.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=9115], [object_number=10, tag_tid=1, entry_tid=9114]]], state=connected] 2024-04-23T21:41:18.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.status 2024-04-23T21:41:18.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:18.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_1 2024-04-23T21:41:18.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:18.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:18.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_1 2024-04-23T21:41:18.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_1 '..*' 2024-04-23T21:41:18.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:18.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:18.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_1 2024-04-23T21:41:18.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:18.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_1.status 2024-04-23T21:41:18.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_1.status 2024-04-23T21:41:18.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:18.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.status 2024-04-23T21:41:18.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_1 2024-04-23T21:41:18.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.status 2024-04-23T21:41:18.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:18.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:18.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:18.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=9115], [object_number=10, tag_tid=1, entry_tid=9114]]], state=connected] 2024-04-23T21:41:18.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=9115], [object_number=10, tag_tid=1, entry_tid=9114]]], state=connected] 2024-04-23T21:41:18.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.status 2024-04-23T21:41:18.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:18.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:18.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:18.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_1 snap 2024-04-23T21:41:18.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:18.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:18.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:41:18.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:41:18.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:41:18.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:41:18.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:41:19.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_1@snap 2024-04-23T21:41:19.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_1': 2024-04-23T21:41:19.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:41:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:41:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:41:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 116eb997a4f0 2024-04-23T21:41:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.116eb997a4f0 2024-04-23T21:41:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:41:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:41:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:41:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:41:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:32:00 2024 2024-04-23T21:41:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:32:00 2024 2024-04-23T21:41:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:32:00 2024 2024-04-23T21:41:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:41:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 116eb997a4f0 2024-04-23T21:41:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:41:19.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:41:19.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 88bf4b3d-bbde-431a-aff8-1c103ba9c96d 2024-04-23T21:41:19.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:41:19.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:19.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_1 snap 2024-04-23T21:41:19.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:19.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:41:19.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:41:19.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:41:19.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.export 2024-04-23T21:41:19.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_1.export 2024-04-23T21:41:19.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_1.export 2024-04-23T21:41:19.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_1@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.export 2024-04-23T21:41:19.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:41:19.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_1@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_1.export 2024-04-23T21:41:20.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:41:20.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_1.export 2024-04-23T21:41:20.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_1.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_1.export 2024-04-23T21:41:20.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:20.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:41:20.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_2 2024-04-23T21:41:20.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_2 snap 2024-04-23T21:41:20.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:41:20.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:20.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:41:20.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:41:20.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_2@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:41:21.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_2 2024-04-23T21:41:21.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:21.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:21.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:41:21.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_2 started 2024-04-23T21:41:21.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:21.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:21.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:41:21.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:41:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:41:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:41:22.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_2 started 2024-04-23T21:41:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:41:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:41:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:41:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:41:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_2 2024-04-23T21:41:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:41:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:41:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:22.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:41:22.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:41:22.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:22.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:41:22.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:22.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:41:22.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:22.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_2 2024-04-23T21:41:22.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:41:22.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:22.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:22.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:22.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:22.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:22.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:22.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:22.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:22.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:22.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:22.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:22.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:22.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:22.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:22.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:41:22.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:22.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:22.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:22.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:41:22.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:22.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:22.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:22.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:22.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:22.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:22.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:22.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:41:22.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:41:22.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:22.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:22.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:22.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:22.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_2 2024-04-23T21:41:22.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 2 closest matches: 2024-04-23T21:41:22.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_22 2024-04-23T21:41:22.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_26 2024-04-23T21:41:22.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:22.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:22.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_2 2024-04-23T21:41:22.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:41:22.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:41:22.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:22.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:22.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:22.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:41:22.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:22.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:22.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:22.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:22.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:22.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:22.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:22.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:41:22.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:41:22.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:22.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:22.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:22.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:22.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_2 2024-04-23T21:41:22.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 4 closest matches: 2024-04-23T21:41:22.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_20 2024-04-23T21:41:22.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_24 2024-04-23T21:41:22.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_25 2024-04-23T21:41:22.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_28 2024-04-23T21:41:22.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:22.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:22.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_2 2024-04-23T21:41:22.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:41:22.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:22.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:41:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:41:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:41:22.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:41:22.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:22.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:22.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:22.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:22.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_2 2024-04-23T21:41:22.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 2 closest matches: 2024-04-23T21:41:22.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_23 2024-04-23T21:41:22.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_27 2024-04-23T21:41:22.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:22.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:22.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_2 2024-04-23T21:41:22.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:41:22.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:22.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:22.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:41:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:41:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:41:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:41:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:41:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:41:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:41:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:41:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:41:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:41:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:41:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:41:22.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:22.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:22.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:22.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_2 2024-04-23T21:41:22.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:41:22.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:41:22.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:41:22.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:41:22.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:41:22.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:41:22.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:41:22.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:22.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_2 2024-04-23T21:41:22.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:41:22.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:41:22.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:22.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:41:22.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:41:22.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_2 2024-04-23T21:41:22.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:41:22.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:41:22.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:22.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:41:22.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:41:22.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:41:22.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:41:22.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:22.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:41:23.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_2 2024-04-23T21:41:23.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:23.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:23.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:41:23.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_2 ']' 2024-04-23T21:41:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_2' 2024-04-23T21:41:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_2 2024-04-23T21:41:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:23.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:23.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_2 2024-04-23T21:41:23.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:23.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:23.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:23.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:23.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:23.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:23.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:23.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:23.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:23.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:23.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:23.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:23.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:23.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:23.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:23.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:23.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:23.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:23.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:23.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_2 2024-04-23T21:41:23.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 2 closest matches: 2024-04-23T21:41:23.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_22 2024-04-23T21:41:23.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_26 2024-04-23T21:41:23.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:23.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:23.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_2 2024-04-23T21:41:23.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:23.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:23.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:23.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:23.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:23.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:23.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:23.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:23.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:23.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:23.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:23.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:23.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:23.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:23.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:23.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:23.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:23.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:23.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:23.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:23.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:23.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:23.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:23.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:23.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:23.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:23.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:23.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:23.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:23.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:23.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:23.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:23.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:23.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_2 2024-04-23T21:41:23.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 4 closest matches: 2024-04-23T21:41:23.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_20 2024-04-23T21:41:23.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_24 2024-04-23T21:41:23.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_25 2024-04-23T21:41:23.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_28 2024-04-23T21:41:23.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:23.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:23.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_2 2024-04-23T21:41:23.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:41:23.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:23.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:23.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:23.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:41:23.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:23.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:23.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:23.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:41:23.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:23.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:23.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:23.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:41:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:41:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:23.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:23.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:23.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:23.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:23.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:23.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:23.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:23.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_2 2024-04-23T21:41:23.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 2 closest matches: 2024-04-23T21:41:23.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_23 2024-04-23T21:41:23.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_27 2024-04-23T21:41:23.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:23.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:23.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_2 2024-04-23T21:41:23.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:41:23.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:23.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:41:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:41:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:41:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:41:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:41:23.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:41:23.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:23.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:23.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:41:23.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:41:23.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:23.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:23.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:23.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:23.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:23.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:23.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:23.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:41:23.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:41:23.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:41:23.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:23.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:23.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:23.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_2 2024-04-23T21:41:23.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:23.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_2 2024-04-23T21:41:23.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:23.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:23.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:41:23.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_2 '' 2024-04-23T21:41:23.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:23.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:23.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:41:23.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:23.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_2.status 2024-04-23T21:41:23.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_2.status 2024-04-23T21:41:23.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:23.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:23.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_2 2024-04-23T21:41:23.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:23.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:41:23.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:41:23.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:23.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15947], [object_number=14, tag_tid=1, entry_tid=15946]]], state=connected] 2024-04-23T21:41:23.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=15948], [object_number=15, tag_tid=1, entry_tid=15947], [object_number=14, tag_tid=1, entry_tid=15946], [object_number=13, tag_tid=1, entry_tid=15945]]], state=connected] 2024-04-23T21:41:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:23.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:41:23.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_2 2024-04-23T21:41:23.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:23.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:23.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:41:23.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_2 '..*' 2024-04-23T21:41:23.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:23.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:23.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:41:23.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:23.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_2.status 2024-04-23T21:41:23.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_2.status 2024-04-23T21:41:23.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:23.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:23.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_2 2024-04-23T21:41:23.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:23.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:41:23.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:41:23.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:23.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15947], [object_number=14, tag_tid=1, entry_tid=15946]]], state=connected] 2024-04-23T21:41:23.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=15948], [object_number=15, tag_tid=1, entry_tid=15947], [object_number=14, tag_tid=1, entry_tid=15946], [object_number=13, tag_tid=1, entry_tid=15945]]], state=connected] 2024-04-23T21:41:23.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:23.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=12, tag_tid=1, entry_tid=15948' 2024-04-23T21:41:23.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=12, tag_tid=1, entry_tid=15948' 2024-04-23T21:41:23.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=12, tag_tid=1, entry_tid=15948' '!=' '' 2024-04-23T21:41:23.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:41:23.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=12, tag_tid=1, entry_tid=15948' = '' 2024-04-23T21:41:23.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=12, tag_tid=1, entry_tid=15948' 2024-04-23T21:41:23.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:41:23.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:41:23.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:23.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:41:23.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=12, tag_tid=1, entry_tid=15948' 2024-04-23T21:41:23.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:41:23.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:23.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:41:23.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:41:23.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:41:23.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:23.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:41:23.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=12, tag_tid=1, entry_tid=15948' 2024-04-23T21:41:23.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:41:23.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:23.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=15948 2024-04-23T21:41:23.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 15948 2024-04-23T21:41:23.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:41:23.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:23.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:41:23.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_2 2024-04-23T21:41:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:41:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_2 ']' 2024-04-23T21:41:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_2' 2024-04-23T21:41:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_2 2024-04-23T21:41:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:23.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:23.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:23.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:23.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:23.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:23.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_2 2024-04-23T21:41:23.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:23.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:23.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:23.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:23.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:23.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:23.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_2 2024-04-23T21:41:23.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 2 closest matches: 2024-04-23T21:41:23.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_22 2024-04-23T21:41:23.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_26 2024-04-23T21:41:23.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:23.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:23.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_2 2024-04-23T21:41:23.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:23.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:23.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:23.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:23.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:23.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:23.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:23.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:23.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:23.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:23.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:23.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:23.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:23.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:23.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:23.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:23.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:23.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:23.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:23.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:23.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:23.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:23.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:23.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:23.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:23.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:23.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:23.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:23.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:23.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:23.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:23.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:23.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:23.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_2 2024-04-23T21:41:23.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 4 closest matches: 2024-04-23T21:41:23.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_20 2024-04-23T21:41:23.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_24 2024-04-23T21:41:23.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_25 2024-04-23T21:41:23.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_28 2024-04-23T21:41:23.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:23.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:23.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_2 2024-04-23T21:41:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:41:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:41:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:41:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:41:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:41:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:24.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:24.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:24.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:24.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_2 2024-04-23T21:41:24.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 2 closest matches: 2024-04-23T21:41:24.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_23 2024-04-23T21:41:24.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_27 2024-04-23T21:41:24.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_2 2024-04-23T21:41:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:41:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:41:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:41:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:41:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:41:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:41:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:41:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:41:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:41:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:41:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:41:24.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:41:24.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:24.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:24.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:24.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_2 2024-04-23T21:41:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_2 2024-04-23T21:41:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:41:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_2 '' 2024-04-23T21:41:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:41:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_2.status 2024-04-23T21:41:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_2.status 2024-04-23T21:41:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:24.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_2 2024-04-23T21:41:24.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:24.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:41:24.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:41:24.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:24.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15947], [object_number=14, tag_tid=1, entry_tid=15946]]], state=connected] 2024-04-23T21:41:24.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=15948], [object_number=15, tag_tid=1, entry_tid=15947], [object_number=14, tag_tid=1, entry_tid=15946], [object_number=13, tag_tid=1, entry_tid=15945]]], state=connected] 2024-04-23T21:41:24.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:41:24.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_2 2024-04-23T21:41:24.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:24.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:24.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:41:24.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_2 '..*' 2024-04-23T21:41:24.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:24.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:24.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:41:24.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:24.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_2.status 2024-04-23T21:41:24.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_2.status 2024-04-23T21:41:24.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:24.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:24.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_2 2024-04-23T21:41:24.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:24.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:41:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:41:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15947], [object_number=14, tag_tid=1, entry_tid=15946]]], state=connected] 2024-04-23T21:41:24.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=15948], [object_number=15, tag_tid=1, entry_tid=15947], [object_number=14, tag_tid=1, entry_tid=15946], [object_number=13, tag_tid=1, entry_tid=15945]]], state=connected] 2024-04-23T21:41:24.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:24.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=12, tag_tid=1, entry_tid=15948' 2024-04-23T21:41:24.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=12, tag_tid=1, entry_tid=15948' 2024-04-23T21:41:24.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=12, tag_tid=1, entry_tid=15948' '!=' 'object_number=12, tag_tid=1, entry_tid=15948' 2024-04-23T21:41:24.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:24.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:41:24.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_2 2024-04-23T21:41:24.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:24.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:41:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_2 ']' 2024-04-23T21:41:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_2' 2024-04-23T21:41:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_2 2024-04-23T21:41:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:24.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:24.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_2 2024-04-23T21:41:24.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:24.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:24.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:24.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:24.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:24.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:24.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:24.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:24.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:24.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:24.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:24.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:24.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:24.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:24.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:24.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:24.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:24.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:24.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:24.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:24.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_2 2024-04-23T21:41:24.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 2 closest matches: 2024-04-23T21:41:24.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_22 2024-04-23T21:41:24.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_26 2024-04-23T21:41:24.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:24.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:24.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_2 2024-04-23T21:41:24.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:24.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:24.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:24.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:24.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:24.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:24.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:24.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:24.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:24.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:24.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:24.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:24.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:24.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:24.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:24.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:24.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:24.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:24.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:24.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:24.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:24.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:24.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:24.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:24.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:24.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:24.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:24.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:24.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:24.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:24.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_2 2024-04-23T21:41:24.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 4 closest matches: 2024-04-23T21:41:24.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_20 2024-04-23T21:41:24.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_24 2024-04-23T21:41:24.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_25 2024-04-23T21:41:24.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_28 2024-04-23T21:41:24.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_2 2024-04-23T21:41:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:41:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:41:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:41:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:41:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:41:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:24.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:24.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:24.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:24.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:24.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:24.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:24.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:24.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:24.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:24.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_2 2024-04-23T21:41:25.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 2 closest matches: 2024-04-23T21:41:25.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_23 2024-04-23T21:41:25.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_27 2024-04-23T21:41:25.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:25.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:25.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_2 2024-04-23T21:41:25.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:41:25.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:25.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:25.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:41:25.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:41:25.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:25.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:25.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:25.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:41:25.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:25.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:41:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:41:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:41:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:41:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:41:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:25.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:25.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:25.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:41:25.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:41:25.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:41:25.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:25.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:25.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:25.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_2 2024-04-23T21:41:25.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:25.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_2 2024-04-23T21:41:25.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:25.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:25.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:41:25.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_2 '' 2024-04-23T21:41:25.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:25.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:25.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:41:25.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:25.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_2.status 2024-04-23T21:41:25.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_2.status 2024-04-23T21:41:25.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:25.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:25.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_2 2024-04-23T21:41:25.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:25.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:41:25.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:41:25.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:25.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15947], [object_number=14, tag_tid=1, entry_tid=15946]]], state=connected] 2024-04-23T21:41:25.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15947], [object_number=14, tag_tid=1, entry_tid=15946]]], state=connected] 2024-04-23T21:41:25.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:25.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:41:25.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_2 2024-04-23T21:41:25.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:25.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:25.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:41:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_2 '..*' 2024-04-23T21:41:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:41:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_2.status 2024-04-23T21:41:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_2.status 2024-04-23T21:41:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:25.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:25.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_2 2024-04-23T21:41:25.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:25.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:41:25.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:41:25.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:25.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15947], [object_number=14, tag_tid=1, entry_tid=15946]]], state=connected] 2024-04-23T21:41:25.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15947], [object_number=14, tag_tid=1, entry_tid=15946]]], state=connected] 2024-04-23T21:41:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.status 2024-04-23T21:41:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:41:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:41:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_2 snap 2024-04-23T21:41:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:25.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:25.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:41:25.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:41:25.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:41:25.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:41:25.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:41:26.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_2@snap 2024-04-23T21:41:26.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_2': 2024-04-23T21:41:26.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:41:26.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:41:26.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:41:26.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10b5c37d321e 2024-04-23T21:41:26.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10b5c37d321e 2024-04-23T21:41:26.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:41:26.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:41:26.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:41:26.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:41:26.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:32:08 2024 2024-04-23T21:41:26.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:32:08 2024 2024-04-23T21:41:26.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:33:09 2024 2024-04-23T21:41:26.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:41:26.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10b5c37d321e 2024-04-23T21:41:26.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:41:26.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:41:26.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 25f37294-47ac-42e3-82f8-619bb2f4f0b0 2024-04-23T21:41:26.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:41:26.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:26.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_2 snap 2024-04-23T21:41:26.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:26.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:41:26.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:41:26.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:41:26.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.export 2024-04-23T21:41:26.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_2.export 2024-04-23T21:41:26.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_2.export 2024-04-23T21:41:26.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_2@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.export 2024-04-23T21:41:26.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:41:26.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_2@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_2.export 2024-04-23T21:41:26.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:41:26.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_2.export 2024-04-23T21:41:26.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_2.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_2.export 2024-04-23T21:41:26.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:26.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:41:26.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_3 2024-04-23T21:41:26.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_3 snap 2024-04-23T21:41:26.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:41:26.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:26.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:41:26.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:41:26.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_3@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:41:29.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_3 2024-04-23T21:41:29.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:29.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:29.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:41:29.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_3 started 2024-04-23T21:41:29.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:29.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:29.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:41:29.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:41:29.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:41:29.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:29.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:41:30.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_3 started 2024-04-23T21:41:30.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:30.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:30.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:41:30.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:41:30.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:41:30.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:41:30.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_3 2024-04-23T21:41:30.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:41:30.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:41:30.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:30.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:30.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:41:30.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:41:30.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:30.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:41:30.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:30.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:41:30.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:30.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_3 2024-04-23T21:41:30.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:41:30.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:30.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:30.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:30.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:30.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:30.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:30.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:30.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:30.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:30.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:30.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:30.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:30.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:30.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:30.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:41:30.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:30.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:30.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:30.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:41:30.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:30.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:30.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:30.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:30.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:30.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:30.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:30.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:41:30.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:41:30.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:30.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:30.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:30.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:30.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_3 2024-04-23T21:41:30.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 3 closest matches: 2024-04-23T21:41:30.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_30 2024-04-23T21:41:30.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_34 2024-04-23T21:41:30.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_38 2024-04-23T21:41:30.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:30.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:30.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_3 2024-04-23T21:41:30.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:41:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:41:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:41:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:41:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:41:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:30.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:30.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:30.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:30.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_3 2024-04-23T21:41:30.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:41:30.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:41:30.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:41:30.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:41:30.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:41:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:41:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:41:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_3 2024-04-23T21:41:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:41:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:41:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:41:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:41:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_3 2024-04-23T21:41:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:41:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:41:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:41:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:41:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:41:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:41:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:41:30.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_3 2024-04-23T21:41:30.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:30.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:30.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:41:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_3 ']' 2024-04-23T21:41:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_3' 2024-04-23T21:41:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_3 2024-04-23T21:41:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:30.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:30.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_3 2024-04-23T21:41:30.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:30.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:30.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:30.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:30.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:30.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:30.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:30.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:30.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:30.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:30.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:30.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:30.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:30.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:30.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:30.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:30.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:30.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:30.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:30.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:30.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_3 2024-04-23T21:41:30.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 3 closest matches: 2024-04-23T21:41:30.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_30 2024-04-23T21:41:30.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_34 2024-04-23T21:41:30.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_38 2024-04-23T21:41:30.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:30.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:30.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_3 2024-04-23T21:41:30.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:30.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:30.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:30.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:30.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:30.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:30.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:30.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:30.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:30.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:30.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:30.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:30.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:30.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:30.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:30.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:30.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:30.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:30.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:30.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:30.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:30.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:30.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:30.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:30.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:30.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:30.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:30.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:30.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:30.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:30.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_3 2024-04-23T21:41:31.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:31.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_3 2024-04-23T21:41:31.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:31.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:31.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:31.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_3 '' 2024-04-23T21:41:31.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:31.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:31.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:31.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:31.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_3.status 2024-04-23T21:41:31.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_3.status 2024-04-23T21:41:31.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:31.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:31.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_3 2024-04-23T21:41:31.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:31.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:31.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:31.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:31.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718]]], state=connected] 2024-04-23T21:41:31.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=8, tag_tid=1, entry_tid=10720], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718], [object_number=9, tag_tid=1, entry_tid=10717]]], state=connected] 2024-04-23T21:41:31.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:31.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:31.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_3 2024-04-23T21:41:31.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:31.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:31.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:31.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_3 '..*' 2024-04-23T21:41:31.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:31.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:31.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:31.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:31.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_3.status 2024-04-23T21:41:31.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_3.status 2024-04-23T21:41:31.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:31.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:31.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_3 2024-04-23T21:41:31.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:31.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:31.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:31.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:31.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718]]], state=connected] 2024-04-23T21:41:31.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=8, tag_tid=1, entry_tid=10720], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718], [object_number=9, tag_tid=1, entry_tid=10717]]], state=connected] 2024-04-23T21:41:31.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:31.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=8, tag_tid=1, entry_tid=10720' 2024-04-23T21:41:31.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=8, tag_tid=1, entry_tid=10720' 2024-04-23T21:41:31.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=8, tag_tid=1, entry_tid=10720' '!=' '' 2024-04-23T21:41:31.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:41:31.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=8, tag_tid=1, entry_tid=10720' = '' 2024-04-23T21:41:31.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=8, tag_tid=1, entry_tid=10720' 2024-04-23T21:41:31.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:31.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:41:31.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:31.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:41:31.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=8, tag_tid=1, entry_tid=10720' 2024-04-23T21:41:31.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:41:31.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:31.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:41:31.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:31.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:41:31.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:31.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:41:31.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=8, tag_tid=1, entry_tid=10720' 2024-04-23T21:41:31.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:41:31.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:31.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=10720 2024-04-23T21:41:31.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 10720 2024-04-23T21:41:31.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:41:31.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:31.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:41:31.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_3 2024-04-23T21:41:31.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:31.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:31.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:41:31.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:31.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_3 ']' 2024-04-23T21:41:31.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_3' 2024-04-23T21:41:31.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_3 2024-04-23T21:41:31.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:31.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:31.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:31.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:31.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:31.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:31.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:31.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:31.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:31.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:31.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_3 2024-04-23T21:41:31.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:31.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:31.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:31.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:31.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:31.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:31.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_3 2024-04-23T21:41:31.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 3 closest matches: 2024-04-23T21:41:31.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_30 2024-04-23T21:41:31.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_34 2024-04-23T21:41:31.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_38 2024-04-23T21:41:31.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_3 2024-04-23T21:41:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:31.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:31.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:31.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:31.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:31.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:31.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:31.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:31.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:31.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:31.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:31.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_3 2024-04-23T21:41:31.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:31.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_3 2024-04-23T21:41:31.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:31.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_3 '' 2024-04-23T21:41:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_3.status 2024-04-23T21:41:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_3.status 2024-04-23T21:41:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:31.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:31.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_3 2024-04-23T21:41:31.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718]]], state=connected] 2024-04-23T21:41:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=8, tag_tid=1, entry_tid=10720], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718], [object_number=9, tag_tid=1, entry_tid=10717]]], state=connected] 2024-04-23T21:41:31.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:31.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:31.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_3 2024-04-23T21:41:31.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:31.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:31.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:31.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_3 '..*' 2024-04-23T21:41:31.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:31.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:31.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:31.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:31.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_3.status 2024-04-23T21:41:31.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_3.status 2024-04-23T21:41:31.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:31.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:31.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_3 2024-04-23T21:41:31.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:31.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:31.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:31.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:31.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718]]], state=connected] 2024-04-23T21:41:31.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=8, tag_tid=1, entry_tid=10720], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718], [object_number=9, tag_tid=1, entry_tid=10717]]], state=connected] 2024-04-23T21:41:31.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:31.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=8, tag_tid=1, entry_tid=10720' 2024-04-23T21:41:31.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=8, tag_tid=1, entry_tid=10720' 2024-04-23T21:41:31.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=8, tag_tid=1, entry_tid=10720' '!=' 'object_number=8, tag_tid=1, entry_tid=10720' 2024-04-23T21:41:31.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:31.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:41:32.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_3 2024-04-23T21:41:32.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:32.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:32.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:41:32.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:32.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_3 ']' 2024-04-23T21:41:32.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_3' 2024-04-23T21:41:32.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_3 2024-04-23T21:41:32.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:32.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:32.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:32.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:32.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:32.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:32.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:32.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:32.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:32.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_3 2024-04-23T21:41:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:32.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:32.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:32.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:32.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:32.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:32.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:32.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:32.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_3 2024-04-23T21:41:32.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 3 closest matches: 2024-04-23T21:41:32.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_30 2024-04-23T21:41:32.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_34 2024-04-23T21:41:32.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_38 2024-04-23T21:41:32.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:32.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:32.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_3 2024-04-23T21:41:32.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:32.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:32.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:32.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:32.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:32.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:32.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:32.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:32.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:32.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:32.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:32.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:32.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:32.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:32.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:32.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:32.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:32.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:32.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:32.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:32.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:32.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:32.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:32.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:32.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:32.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:32.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:32.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:32.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:32.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:32.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:32.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:32.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:32.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_3 2024-04-23T21:41:32.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:32.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_3 2024-04-23T21:41:32.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:32.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:32.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:32.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_3 '' 2024-04-23T21:41:32.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:32.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:32.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:32.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:32.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_3.status 2024-04-23T21:41:32.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_3.status 2024-04-23T21:41:32.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:32.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:32.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_3 2024-04-23T21:41:32.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:32.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:32.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:32.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:32.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718]]], state=connected] 2024-04-23T21:41:32.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=8, tag_tid=1, entry_tid=10720], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718], [object_number=9, tag_tid=1, entry_tid=10717]]], state=connected] 2024-04-23T21:41:32.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:32.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:32.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_3 2024-04-23T21:41:32.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:32.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_3 '..*' 2024-04-23T21:41:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_3.status 2024-04-23T21:41:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_3.status 2024-04-23T21:41:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_3 2024-04-23T21:41:32.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:32.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:32.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:32.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:32.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718]]], state=connected] 2024-04-23T21:41:32.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=8, tag_tid=1, entry_tid=10720], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718], [object_number=9, tag_tid=1, entry_tid=10717]]], state=connected] 2024-04-23T21:41:32.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:32.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=8, tag_tid=1, entry_tid=10720' 2024-04-23T21:41:32.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=8, tag_tid=1, entry_tid=10720' 2024-04-23T21:41:32.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=8, tag_tid=1, entry_tid=10720' '!=' 'object_number=8, tag_tid=1, entry_tid=10720' 2024-04-23T21:41:32.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:32.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:41:33.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_3 2024-04-23T21:41:33.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:33.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:41:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_3 ']' 2024-04-23T21:41:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_3' 2024-04-23T21:41:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_3 2024-04-23T21:41:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:33.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:33.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:33.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_3 2024-04-23T21:41:33.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:33.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:33.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:33.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:33.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:33.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:33.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:33.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:33.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:33.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:33.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:33.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:33.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:33.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:33.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:33.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:33.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:33.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:33.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:33.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:33.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_3 2024-04-23T21:41:33.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 3 closest matches: 2024-04-23T21:41:33.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_30 2024-04-23T21:41:33.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_34 2024-04-23T21:41:33.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_38 2024-04-23T21:41:33.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_3 2024-04-23T21:41:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:33.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:33.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:33.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:33.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:33.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:33.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:33.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:33.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:33.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:33.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:33.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:33.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_3 2024-04-23T21:41:33.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:33.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_3 2024-04-23T21:41:33.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:33.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:33.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:33.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_3 '' 2024-04-23T21:41:33.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:33.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:33.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:33.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:33.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_3.status 2024-04-23T21:41:33.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_3.status 2024-04-23T21:41:33.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_3 2024-04-23T21:41:33.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:33.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:33.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:33.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:33.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718]]], state=connected] 2024-04-23T21:41:33.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=8, tag_tid=1, entry_tid=10720], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718], [object_number=9, tag_tid=1, entry_tid=10717]]], state=connected] 2024-04-23T21:41:33.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:33.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:33.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_3 2024-04-23T21:41:33.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:33.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:33.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:33.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_3 '..*' 2024-04-23T21:41:33.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:33.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:33.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:33.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:33.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_3.status 2024-04-23T21:41:33.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_3.status 2024-04-23T21:41:33.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:33.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:33.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_3 2024-04-23T21:41:33.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:33.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:33.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:33.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:33.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718]]], state=connected] 2024-04-23T21:41:33.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=8, tag_tid=1, entry_tid=10720], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718], [object_number=9, tag_tid=1, entry_tid=10717]]], state=connected] 2024-04-23T21:41:33.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:33.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=8, tag_tid=1, entry_tid=10720' 2024-04-23T21:41:33.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=8, tag_tid=1, entry_tid=10720' 2024-04-23T21:41:33.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=8, tag_tid=1, entry_tid=10720' '!=' 'object_number=8, tag_tid=1, entry_tid=10720' 2024-04-23T21:41:33.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:33.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:41:35.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_3 2024-04-23T21:41:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:41:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_3 ']' 2024-04-23T21:41:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_3' 2024-04-23T21:41:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_3 2024-04-23T21:41:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:35.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:35.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:35.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:35.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:35.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:35.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:35.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_3 2024-04-23T21:41:35.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:35.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:35.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:35.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:35.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:35.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:35.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:35.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:35.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:35.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:35.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:35.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:35.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:35.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:35.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:35.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:35.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:35.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:35.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:35.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:35.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_3 2024-04-23T21:41:35.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 3 closest matches: 2024-04-23T21:41:35.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_30 2024-04-23T21:41:35.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_34 2024-04-23T21:41:35.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_38 2024-04-23T21:41:35.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:35.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:35.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_3 2024-04-23T21:41:35.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:35.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:35.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:35.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:35.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:35.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:35.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:35.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:35.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:35.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:35.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:35.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:35.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:35.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:35.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:35.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:35.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:35.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:35.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:35.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:35.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:35.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:35.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:35.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:35.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:35.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:35.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:35.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:35.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:35.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_3 2024-04-23T21:41:35.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:35.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_3 2024-04-23T21:41:35.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:35.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:35.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:35.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_3 '' 2024-04-23T21:41:35.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:35.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:35.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:35.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:35.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_3.status 2024-04-23T21:41:35.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_3.status 2024-04-23T21:41:35.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:35.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:35.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_3 2024-04-23T21:41:35.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:35.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:35.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:35.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:35.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718]]], state=connected] 2024-04-23T21:41:35.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718]]], state=connected] 2024-04-23T21:41:35.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:35.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:35.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_3 2024-04-23T21:41:35.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:35.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:35.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:35.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_3 '..*' 2024-04-23T21:41:35.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:35.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:41:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_3.status 2024-04-23T21:41:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_3.status 2024-04-23T21:41:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_3 2024-04-23T21:41:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:35.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:35.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:35.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:35.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718]]], state=connected] 2024-04-23T21:41:35.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10719], [object_number=10, tag_tid=1, entry_tid=10718]]], state=connected] 2024-04-23T21:41:35.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.status 2024-04-23T21:41:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_3 snap 2024-04-23T21:41:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:41:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:41:35.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:41:35.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:41:35.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:41:36.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_3@snap 2024-04-23T21:41:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_3': 2024-04-23T21:41:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:41:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:41:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:41:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 119212fea226 2024-04-23T21:41:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.119212fea226 2024-04-23T21:41:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:41:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:41:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:41:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:41:36.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:32:21 2024 2024-04-23T21:41:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:32:21 2024 2024-04-23T21:41:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:33:22 2024 2024-04-23T21:41:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:41:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 119212fea226 2024-04-23T21:41:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:41:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:41:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 47db0c8c-b524-4797-905a-f78a4453a413 2024-04-23T21:41:36.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:41:36.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:36.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_3 snap 2024-04-23T21:41:36.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:36.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:41:36.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:41:36.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:41:36.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.export 2024-04-23T21:41:36.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_3.export 2024-04-23T21:41:36.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_3.export 2024-04-23T21:41:36.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_3@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.export 2024-04-23T21:41:36.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:41:36.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_3@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_3.export 2024-04-23T21:41:36.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:41:36.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_3.export 2024-04-23T21:41:36.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_3.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_3.export 2024-04-23T21:41:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:41:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_4 2024-04-23T21:41:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_4 snap 2024-04-23T21:41:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:41:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:41:36.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:41:36.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_4@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:41:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_4 2024-04-23T21:41:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:41:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_4 started 2024-04-23T21:41:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:41:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:41:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:41:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:41:40.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_4 started 2024-04-23T21:41:40.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:40.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:40.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:41:40.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:41:40.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:41:40.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:41:40.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_4 2024-04-23T21:41:40.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:41:40.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:41:40.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:40.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:40.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:41:40.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:41:40.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:40.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:41:40.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:40.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:41:40.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:40.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_4 2024-04-23T21:41:40.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:41:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:40.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:41:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:41:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:41:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:41:40.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:40.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:40.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:40.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:40.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_4 2024-04-23T21:41:40.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 1 closest matches: 2024-04-23T21:41:40.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_42 2024-04-23T21:41:40.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_4 2024-04-23T21:41:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:41:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:40.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:40.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:40.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:40.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:40.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:40.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:40.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:40.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:40.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:40.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:40.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:40.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:40.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:41:40.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:40.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:40.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:40.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:41:40.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:40.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:40.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:40.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:40.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:40.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:40.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:40.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:41:40.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:41:40.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:40.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:40.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:40.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:40.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_4 2024-04-23T21:41:40.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 2 closest matches: 2024-04-23T21:41:40.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_40 2024-04-23T21:41:40.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_47 2024-04-23T21:41:40.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:40.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:40.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_4 2024-04-23T21:41:40.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:41:40.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:40.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:40.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:40.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:40.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:40.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:40.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:40.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:40.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:40.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:40.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:40.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:41:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:41:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:40.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:40.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:40.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:40.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:40.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:40.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:40.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:41:40.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:41:40.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:40.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:40.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:40.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:40.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_4 2024-04-23T21:41:40.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 4 closest matches: 2024-04-23T21:41:40.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_41 2024-04-23T21:41:40.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_43 2024-04-23T21:41:40.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_46 2024-04-23T21:41:40.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status mirror/image_49 2024-04-23T21:41:40.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:40.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:40.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_4 2024-04-23T21:41:40.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:41:40.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:40.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:40.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:41:40.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:41:40.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:40.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:40.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:40.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:40.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:40.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:40.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:40.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:41:40.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:41:40.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:41:40.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:41:40.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:40.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:40.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:41:40.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:41:40.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:40.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:40.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:40.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:41:40.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:40.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:40.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:40.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:41:40.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:41:40.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:41:40.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:40.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:40.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:40.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_4 2024-04-23T21:41:40.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:41:40.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:41:40.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:41:40.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:41:40.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:41:40.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:41:40.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:41:40.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:40.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_4 2024-04-23T21:41:40.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:41:40.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:41:40.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:40.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:41:40.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:41:40.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_4 2024-04-23T21:41:40.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:41:40.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:41:40.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:40.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:41:40.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:41:40.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:41:40.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:41:40.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:40.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:41:40.881 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8426065377582322) > weight (0.25) 2024-04-23T21:41:40.881 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.355825377185638) > weight (0.25) 2024-04-23T21:41:40.881 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9693871384216628) > weight (0.25) 2024-04-23T21:41:40.881 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2024-04-23T21:41:40.881 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2024-04-23T21:41:40.882 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2024-04-23T21:41:40.882 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:41:40.883+0000 7f8a6cd36640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 64162) UID: 1000 2024-04-23T21:41:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_4 2024-04-23T21:41:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:41:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_4 ']' 2024-04-23T21:41:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_4' 2024-04-23T21:41:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_4 2024-04-23T21:41:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:41.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:41.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:41.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:41.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:41.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:41.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:41.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:41.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:41.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_4 2024-04-23T21:41:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:41.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:41.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:41.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:41.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:41.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:41.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 1 closest matches: 2024-04-23T21:41:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_42 2024-04-23T21:41:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:41.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:41.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_4 2024-04-23T21:41:41.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:41.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:41.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:41.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:41.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:41.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:41.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:41.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:41.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:41.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:41.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:41.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:41.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:41.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:41.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:41.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:41.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:41.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:41.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:41.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:41.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:41.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 2 closest matches: 2024-04-23T21:41:41.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_40 2024-04-23T21:41:41.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_47 2024-04-23T21:41:41.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:41.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_4 2024-04-23T21:41:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:41:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:41:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:41:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:41:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:41:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:41.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:41.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:41.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:41.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:41.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:41.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:41.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:41.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:41.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:41.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:41.284 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9251721582077644) > weight (0.25) 2024-04-23T21:41:41.285 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7384291356718181) > weight (0.25) 2024-04-23T21:41:41.285 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4836823262419855) > weight (0.25) 2024-04-23T21:41:41.285 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9612844751069255) > weight (0.25) 2024-04-23T21:41:41.285 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2024-04-23T21:41:41.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 4 closest matches: 2024-04-23T21:41:41.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_41 2024-04-23T21:41:41.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_43 2024-04-23T21:41:41.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_46 2024-04-23T21:41:41.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_49 2024-04-23T21:41:41.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:41.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_4 2024-04-23T21:41:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:41:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:41:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:41:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:41:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:41:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:41:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:41:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:41:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:41:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:41:41.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:41:41.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:41:41.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:41.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:41.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:41.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:41.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:41.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_4 2024-04-23T21:41:41.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:41.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:41.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:41.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_4 '' 2024-04-23T21:41:41.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:41.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:41.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:41.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:41.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_4.status 2024-04-23T21:41:41.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_4.status 2024-04-23T21:41:41.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:41.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:41.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_4 2024-04-23T21:41:41.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:41.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:41.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:41.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:41.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:41.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11537], [object_number=8, tag_tid=1, entry_tid=11536], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:41.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:41.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:41.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_4 2024-04-23T21:41:41.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:41.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:41.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:41.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_4 '..*' 2024-04-23T21:41:41.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:41.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:41.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:41.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:41.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_4.status 2024-04-23T21:41:41.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_4.status 2024-04-23T21:41:41.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:41.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:41.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_4 2024-04-23T21:41:41.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:41.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:41.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:41.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:41.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:41.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11537], [object_number=8, tag_tid=1, entry_tid=11536], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:41.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:41.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=1, entry_tid=11537' 2024-04-23T21:41:41.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=1, entry_tid=11537' 2024-04-23T21:41:41.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=11537' '!=' '' 2024-04-23T21:41:41.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:41:41.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=11537' = '' 2024-04-23T21:41:41.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=9, tag_tid=1, entry_tid=11537' 2024-04-23T21:41:41.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:41.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:41:41.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:41.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:41:41.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=1, entry_tid=11537' 2024-04-23T21:41:41.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:41:41.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:41.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:41:41.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:41.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:41:41.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:41.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:41:41.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=1, entry_tid=11537' 2024-04-23T21:41:41.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:41:41.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:41.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=11537 2024-04-23T21:41:41.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 11537 2024-04-23T21:41:41.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:41:41.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:41.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:41:41.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_4 2024-04-23T21:41:41.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:41.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:41.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:41:41.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:41.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_4 ']' 2024-04-23T21:41:41.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_4' 2024-04-23T21:41:41.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_4 2024-04-23T21:41:41.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:41.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:41.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:41.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:41.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:41.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:41.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:41.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:41.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:41.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:41.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_4 2024-04-23T21:41:41.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:41.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:41.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:41.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:41.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:41.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:41.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:41.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:41.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:41.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:41.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:41.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:41.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:41.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:41.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:41.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:41.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:41.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:41.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:41.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:41.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:41.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:41.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:41.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:41.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:41.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:41.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:41.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:41.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:41.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:41.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:41.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:41.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:41.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:41.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 1 closest matches: 2024-04-23T21:41:41.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_42 2024-04-23T21:41:41.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:41.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:41.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_4 2024-04-23T21:41:41.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:41.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:41.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:41.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:41.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:41.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:41.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:41.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:41.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:41.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:41.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:41.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:41.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:41.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:41.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:41.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:41.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:41.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:41.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:41.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:41.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:41.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:41.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:41.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:41.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:41.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:41.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:41.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:41.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:41.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:41.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:41.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:41.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:41.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:41.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 2 closest matches: 2024-04-23T21:41:41.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_40 2024-04-23T21:41:41.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_47 2024-04-23T21:41:41.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:41.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:41.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_4 2024-04-23T21:41:41.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:41:41.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:41.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:41.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:41.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:41:41.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:41.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:41.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:41.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:41:41.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:41:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:41:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:41.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:41.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:41.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:41.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:41.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:41.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:41.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:41.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:41.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:41.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:42.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 4 closest matches: 2024-04-23T21:41:42.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_41 2024-04-23T21:41:42.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_43 2024-04-23T21:41:42.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_46 2024-04-23T21:41:42.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_49 2024-04-23T21:41:42.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_4 2024-04-23T21:41:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:41:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:41:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:41:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:41:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:41:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:41:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:41:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:41:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:41:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:42.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:42.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:42.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:42.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:41:42.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:41:42.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:41:42.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:42.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:42.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:42.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:42.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:42.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_4 2024-04-23T21:41:42.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:42.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:42.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:42.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_4 '' 2024-04-23T21:41:42.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:42.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:42.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:42.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:42.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_4.status 2024-04-23T21:41:42.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_4.status 2024-04-23T21:41:42.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:42.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:42.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_4 2024-04-23T21:41:42.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:42.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:42.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:42.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:42.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:42.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11537], [object_number=8, tag_tid=1, entry_tid=11536], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:42.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:42.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:42.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_4 2024-04-23T21:41:42.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:42.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:42.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:42.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_4 '..*' 2024-04-23T21:41:42.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:42.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:42.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:42.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:42.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_4.status 2024-04-23T21:41:42.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_4.status 2024-04-23T21:41:42.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:42.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:42.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_4 2024-04-23T21:41:42.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:42.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:42.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:42.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:42.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:42.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11537], [object_number=8, tag_tid=1, entry_tid=11536], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:42.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:42.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=1, entry_tid=11537' 2024-04-23T21:41:42.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=1, entry_tid=11537' 2024-04-23T21:41:42.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=11537' '!=' 'object_number=9, tag_tid=1, entry_tid=11537' 2024-04-23T21:41:42.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:42.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:41:42.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_4 2024-04-23T21:41:42.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:42.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:42.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:41:42.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:42.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_4 ']' 2024-04-23T21:41:42.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_4' 2024-04-23T21:41:42.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_4 2024-04-23T21:41:42.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:42.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:42.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:42.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:42.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:42.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:42.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:42.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:42.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:42.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:42.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_4 2024-04-23T21:41:42.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:42.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:42.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:42.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:42.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:42.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:42.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:42.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:42.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:42.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:42.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:42.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:42.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:42.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:42.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:42.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:42.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:42.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:42.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:42.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:42.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:42.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:42.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:42.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:42.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:42.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:42.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:42.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:42.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:42.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:42.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:42.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:42.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:42.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:42.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 1 closest matches: 2024-04-23T21:41:42.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_42 2024-04-23T21:41:42.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:42.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:42.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_4 2024-04-23T21:41:42.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:42.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:42.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:42.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:42.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:42.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:42.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:42.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:42.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:42.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:42.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:42.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:42.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:42.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:42.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:42.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:42.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:42.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:42.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:42.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:42.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:42.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:42.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:42.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:42.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:42.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:42.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:42.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:42.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:42.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:42.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:42.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:42.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:42.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:42.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 2 closest matches: 2024-04-23T21:41:42.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_40 2024-04-23T21:41:42.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_47 2024-04-23T21:41:42.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:42.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:42.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_4 2024-04-23T21:41:42.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:41:42.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:42.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:42.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:42.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:41:42.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:42.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:42.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:42.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:41:42.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:42.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:42.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:42.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:41:42.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:42.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:42.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:41:42.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:42.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:42.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:42.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:42.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:42.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:42.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:42.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:42.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:42.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:42.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:42.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:42.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:42.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:42.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:42.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:42.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:42.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:42.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 4 closest matches: 2024-04-23T21:41:42.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_41 2024-04-23T21:41:42.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_43 2024-04-23T21:41:42.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_46 2024-04-23T21:41:42.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_49 2024-04-23T21:41:42.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:42.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:42.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_4 2024-04-23T21:41:42.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:41:42.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:42.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:42.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:41:42.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:41:42.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:42.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:42.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:42.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:41:42.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:42.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:42.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:42.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:41:42.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:42.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:41:42.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:41:42.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:42.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:42.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:41:42.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:41:42.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:42.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:42.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:42.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:42.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:42.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:42.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:42.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:41:42.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:41:42.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:41:42.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:42.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:42.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:42.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_4 2024-04-23T21:41:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_4 '' 2024-04-23T21:41:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:43.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:43.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:43.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_4.status 2024-04-23T21:41:43.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:43.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_4.status 2024-04-23T21:41:43.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:43.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_4 2024-04-23T21:41:43.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:43.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:43.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:43.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:43.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:43.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11537], [object_number=8, tag_tid=1, entry_tid=11536], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:43.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:43.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:43.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_4 2024-04-23T21:41:43.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:43.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:43.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:43.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_4 '..*' 2024-04-23T21:41:43.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:43.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:43.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:43.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:43.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_4.status 2024-04-23T21:41:43.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_4.status 2024-04-23T21:41:43.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:43.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:43.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_4 2024-04-23T21:41:43.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:43.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:43.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11537], [object_number=8, tag_tid=1, entry_tid=11536], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:43.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:43.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=1, entry_tid=11537' 2024-04-23T21:41:43.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=1, entry_tid=11537' 2024-04-23T21:41:43.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=11537' '!=' 'object_number=9, tag_tid=1, entry_tid=11537' 2024-04-23T21:41:43.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:43.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:41:43.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_4 2024-04-23T21:41:43.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:43.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:43.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:41:43.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:43.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_4 ']' 2024-04-23T21:41:43.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_4' 2024-04-23T21:41:43.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_4 2024-04-23T21:41:43.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:43.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:43.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:43.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:43.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:43.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:43.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:43.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:43.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:43.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:43.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_4 2024-04-23T21:41:43.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:43.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:43.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:43.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:43.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:43.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:43.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:43.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:43.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:43.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:43.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:43.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:43.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:43.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:43.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:43.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:44.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:44.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:44.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:44.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:44.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 1 closest matches: 2024-04-23T21:41:44.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_42 2024-04-23T21:41:44.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:44.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:44.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_4 2024-04-23T21:41:44.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:44.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:44.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:44.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:44.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:44.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:44.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:44.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:44.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:44.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:44.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:44.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:44.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:44.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:44.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:44.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:44.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:44.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:44.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:44.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:44.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:44.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:44.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:44.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:44.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:44.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:44.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:44.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:44.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:44.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:44.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:44.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:44.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:44.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:44.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 2 closest matches: 2024-04-23T21:41:44.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_40 2024-04-23T21:41:44.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_47 2024-04-23T21:41:44.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:44.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:44.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_4 2024-04-23T21:41:44.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:41:44.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:44.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:44.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:44.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:41:44.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:44.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:44.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:44.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:41:44.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:44.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:44.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:44.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:41:44.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:44.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:44.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:41:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:44.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:44.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:44.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:44.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:44.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:44.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 4 closest matches: 2024-04-23T21:41:44.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_41 2024-04-23T21:41:44.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_43 2024-04-23T21:41:44.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_46 2024-04-23T21:41:44.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_49 2024-04-23T21:41:44.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:44.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:44.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_4 2024-04-23T21:41:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:41:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:41:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:41:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:41:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:41:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:41:44.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:41:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:41:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:41:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:41:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:41:44.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:41:44.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:44.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:44.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:44.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:44.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:44.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_4 2024-04-23T21:41:44.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:44.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:44.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:44.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_4 '' 2024-04-23T21:41:44.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:44.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:44.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:44.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:44.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_4.status 2024-04-23T21:41:44.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:44.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_4.status 2024-04-23T21:41:44.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:44.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_4 2024-04-23T21:41:44.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:44.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:44.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:44.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:44.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:44.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11537], [object_number=8, tag_tid=1, entry_tid=11536], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:44.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:44.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:44.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_4 2024-04-23T21:41:44.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:44.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:44.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:44.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_4 '..*' 2024-04-23T21:41:44.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:44.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:44.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:44.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:44.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_4.status 2024-04-23T21:41:44.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_4.status 2024-04-23T21:41:44.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:44.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:44.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_4 2024-04-23T21:41:44.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:44.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:44.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:44.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:44.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:44.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11537], [object_number=8, tag_tid=1, entry_tid=11536], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:44.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:44.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=1, entry_tid=11537' 2024-04-23T21:41:44.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=1, entry_tid=11537' 2024-04-23T21:41:44.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=11537' '!=' 'object_number=9, tag_tid=1, entry_tid=11537' 2024-04-23T21:41:44.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:44.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:41:46.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_4 2024-04-23T21:41:46.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:46.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:46.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:41:46.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:46.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_4 ']' 2024-04-23T21:41:46.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_4' 2024-04-23T21:41:46.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_4 2024-04-23T21:41:46.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:46.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:46.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:46.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:46.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:46.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:46.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:46.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:46.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:46.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:46.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_4 2024-04-23T21:41:46.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:46.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:46.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:46.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:46.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:46.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:46.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:46.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:46.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:46.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:46.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:46.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:46.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:46.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:46.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:46.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:46.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:46.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:46.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:46.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:46.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:46.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:46.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:46.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:46.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:46.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:46.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:46.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:46.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:46.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:46.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:46.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:46.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:46.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:46.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 1 closest matches: 2024-04-23T21:41:46.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_42 2024-04-23T21:41:46.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:46.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:46.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_4 2024-04-23T21:41:46.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:46.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:46.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:46.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:46.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:46.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:46.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 2 closest matches: 2024-04-23T21:41:46.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_40 2024-04-23T21:41:46.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_47 2024-04-23T21:41:46.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_4 2024-04-23T21:41:46.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:41:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:41:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:41:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:41:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:46.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:46.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:41:46.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:46.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:46.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:46.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:46.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:46.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:46.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:46.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:46.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:46.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:46.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:46.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:46.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:46.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:46.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:46.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:46.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:46.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:no valid command found; 4 closest matches: 2024-04-23T21:41:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_41 2024-04-23T21:41:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_43 2024-04-23T21:41:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_46 2024-04-23T21:41:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush mirror/image_49 2024-04-23T21:41:46.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:46.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:46.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_4 2024-04-23T21:41:46.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:41:46.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:46.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:46.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:41:46.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:41:46.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:46.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:46.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:46.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:41:46.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:46.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:46.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:41:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:41:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:41:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:41:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:41:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:41:46.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:41:46.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:46.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:46.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:41:46.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:41:46.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:41:46.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:46.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:46.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:41:46.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_4 2024-04-23T21:41:46.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:46.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_4 2024-04-23T21:41:46.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:46.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:46.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:46.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_4 '' 2024-04-23T21:41:46.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:46.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:46.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:46.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:46.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_4.status 2024-04-23T21:41:46.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_4.status 2024-04-23T21:41:46.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:46.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:46.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_4 2024-04-23T21:41:46.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:46.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:46.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:46.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:46.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:46.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:46.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:46.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:46.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_4 2024-04-23T21:41:46.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:46.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:46.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:46.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_4 '..*' 2024-04-23T21:41:46.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:46.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:46.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:41:46.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:46.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_4.status 2024-04-23T21:41:46.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_4.status 2024-04-23T21:41:46.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:46.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:46.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_4 2024-04-23T21:41:46.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:46.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:41:46.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:41:46.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:46.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:46.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11535], [object_number=10, tag_tid=1, entry_tid=11534]]], state=connected] 2024-04-23T21:41:46.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.status 2024-04-23T21:41:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:41:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_4 snap 2024-04-23T21:41:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:41:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:41:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:41:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:41:46.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:41:46.882 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2024-04-23T21:41:46.882 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:41:46.882 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2024-04-23T21:41:46.882 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2024-04-23T21:41:46.883 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2024-04-23T21:41:46.885 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2024-04-23T21:41:46.885 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2024-04-23T21:41:47.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_4@snap 2024-04-23T21:41:47.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_4': 2024-04-23T21:41:47.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:41:47.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:41:47.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:41:47.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 11b8f48000e2 2024-04-23T21:41:47.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.11b8f48000e2 2024-04-23T21:41:47.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:41:47.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:41:47.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:41:47.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:41:47.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:32:30 2024 2024-04-23T21:41:47.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:32:30 2024 2024-04-23T21:41:47.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:33:31 2024 2024-04-23T21:41:47.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:41:47.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 11b8f48000e2 2024-04-23T21:41:47.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:41:47.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:41:47.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 30dae87e-11d9-4a7a-ad0e-6e4c0364c739 2024-04-23T21:41:47.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:41:47.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:47.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_4 snap 2024-04-23T21:41:47.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:47.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:41:47.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:41:47.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:41:47.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.export 2024-04-23T21:41:47.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_4.export 2024-04-23T21:41:47.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_4.export 2024-04-23T21:41:47.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_4@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.export 2024-04-23T21:41:48.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:41:48.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_4@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_4.export 2024-04-23T21:41:48.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:41:48.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_4.export 2024-04-23T21:41:48.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_4.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_4.export 2024-04-23T21:41:48.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:48.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:41:48.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_5 2024-04-23T21:41:48.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_5 snap 2024-04-23T21:41:48.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:41:48.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:48.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:41:48.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:41:48.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_5@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:41:49.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_5 2024-04-23T21:41:49.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:49.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:49.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:41:49.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_5 started 2024-04-23T21:41:49.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:49.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:49.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:41:49.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:41:49.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:41:49.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:49.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:41:50.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_5 started 2024-04-23T21:41:50.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:50.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:50.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:41:50.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:41:50.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:41:50.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:41:50.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_5 2024-04-23T21:41:50.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:41:50.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:41:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:41:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:41:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:50.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:41:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:41:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_5 2024-04-23T21:41:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:41:50.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:50.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:50.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:50.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:50.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:50.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:50.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:50.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:50.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:50.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:50.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:50.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:50.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:50.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:50.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:41:50.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:50.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:50.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:50.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:41:50.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:50.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:50.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:50.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:50.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:50.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:50.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:50.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:41:50.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:41:50.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:50.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:50.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:50.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:50.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_5 2024-04-23T21:41:51.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_5'] 2024-04-23T21:41:51.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:41:51.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:51.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:51.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_5 2024-04-23T21:41:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:41:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:51.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:41:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:41:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:51.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:51.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:51.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:51.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:41:51.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:41:51.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:51.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:51.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:51.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:51.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_5 2024-04-23T21:41:51.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_5'] 2024-04-23T21:41:51.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:41:51.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:51.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:51.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_5 2024-04-23T21:41:51.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:41:51.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:41:51.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:51.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:51.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:51.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:41:51.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:51.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:51.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:51.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:51.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:41:51.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:51.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:51.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:51.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:51.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:41:51.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:41:51.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:41:51.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:51.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:41:51.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:41:51.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:41:51.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:41:51.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:51.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:51.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:41:51.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:41:51.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:41:51.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:41:51.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:51.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:51.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:51.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:51.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_5 2024-04-23T21:41:51.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:41:51.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:41:51.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:41:51.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:41:51.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:41:51.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:41:51.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:41:51.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:51.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_5 2024-04-23T21:41:51.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:41:51.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:41:51.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:51.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:41:51.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:41:51.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_5 2024-04-23T21:41:51.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:41:51.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:41:51.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:51.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:41:51.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:41:51.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:41:51.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:41:51.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:51.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:41:51.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_5 2024-04-23T21:41:51.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:51.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:51.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:41:51.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:51.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_5 ']' 2024-04-23T21:41:51.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_5' 2024-04-23T21:41:51.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_5 2024-04-23T21:41:51.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:51.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_5 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:51.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:51.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:51.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:51.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:51.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:51.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:51.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:51.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:51.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:51.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:51.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:51.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:51.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:51.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:51.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_5'] 2024-04-23T21:41:51.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:41:51.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:51.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:51.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_5 2024-04-23T21:41:51.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:51.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:51.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:51.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:51.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:51.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:51.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:51.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:51.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:51.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:51.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:51.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:51.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:51.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:51.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:51.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:51.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:51.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:51.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:51.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:51.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_5'] 2024-04-23T21:41:51.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:41:51.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:51.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:51.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_5 2024-04-23T21:41:51.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:41:51.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:51.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:51.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:51.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:41:51.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:51.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:51.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:51.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:41:51.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:51.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:51.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:51.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:41:51.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:51.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:51.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:41:51.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:51.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:51.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:51.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:51.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:51.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:51.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:51.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:51.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:51.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:51.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:51.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:51.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:51.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:51.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:51.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:51.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:51.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:51.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:51.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_5 2024-04-23T21:41:51.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:51.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:51.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:51.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_5 '' 2024-04-23T21:41:51.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:51.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:51.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:51.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:51.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_5.status 2024-04-23T21:41:51.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_5.status 2024-04-23T21:41:51.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:51.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:51.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_5 2024-04-23T21:41:51.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:51.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:41:51.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:41:51.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:51.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:51.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=8393], [object_number=4, tag_tid=1, entry_tid=8392], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:51.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:51.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:41:51.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_5 2024-04-23T21:41:51.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:51.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:51.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:51.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_5 '..*' 2024-04-23T21:41:51.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:51.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:51.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:51.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:51.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_5.status 2024-04-23T21:41:51.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_5.status 2024-04-23T21:41:51.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:51.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:51.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_5 2024-04-23T21:41:51.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:51.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:41:51.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:41:51.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:51.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:51.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=8393], [object_number=4, tag_tid=1, entry_tid=8392], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:51.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:51.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:51.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:51.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=8393' '!=' '' 2024-04-23T21:41:51.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:41:51.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=8393' = '' 2024-04-23T21:41:51.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:51.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:41:51.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:41:51.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:51.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:41:51.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:51.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:41:51.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:51.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:41:51.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:41:51.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:41:51.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:51.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:41:51.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:51.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:41:51.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:41:51.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=8393 2024-04-23T21:41:51.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 8393 2024-04-23T21:41:51.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:41:51.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:51.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:41:52.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_5 2024-04-23T21:41:52.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:52.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:52.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:41:52.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:52.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_5 ']' 2024-04-23T21:41:52.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_5' 2024-04-23T21:41:52.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_5 2024-04-23T21:41:52.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:52.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:52.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:52.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:52.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:52.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:52.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:52.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:52.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:52.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_5 2024-04-23T21:41:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:52.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:52.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:52.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:52.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:52.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:52.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:52.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:52.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:52.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:52.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:52.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_5'] 2024-04-23T21:41:52.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:41:52.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:52.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:52.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_5 2024-04-23T21:41:52.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:52.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:52.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:52.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:52.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:52.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:52.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:52.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:52.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:52.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:52.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:52.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:52.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:52.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_5'] 2024-04-23T21:41:52.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:41:52.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_5 2024-04-23T21:41:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:41:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:41:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:52.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:41:52.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:52.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:52.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:52.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:41:52.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:52.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:52.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:41:52.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:52.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:52.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:52.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:52.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:52.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:52.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:52.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:52.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:52.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:52.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:52.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:52.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:52.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:52.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:52.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:52.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:52.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:52.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:52.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_5 2024-04-23T21:41:52.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:52.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:52.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:52.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_5 '' 2024-04-23T21:41:52.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:52.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:52.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:52.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:52.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_5.status 2024-04-23T21:41:52.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_5.status 2024-04-23T21:41:52.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:52.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:52.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_5 2024-04-23T21:41:52.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:52.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:41:52.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:41:52.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:52.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:52.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=8393], [object_number=4, tag_tid=1, entry_tid=8392], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:52.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:52.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:41:52.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_5 2024-04-23T21:41:52.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:52.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:52.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:52.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_5 '..*' 2024-04-23T21:41:52.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:52.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:52.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:52.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:52.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_5.status 2024-04-23T21:41:52.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_5.status 2024-04-23T21:41:52.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:52.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:52.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_5 2024-04-23T21:41:52.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:52.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:41:52.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:41:52.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:52.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:52.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=8393], [object_number=4, tag_tid=1, entry_tid=8392], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:52.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:52.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:52.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:52.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=8393' '!=' 'object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:52.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:52.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:41:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_5 2024-04-23T21:41:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:41:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_5 ']' 2024-04-23T21:41:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_5' 2024-04-23T21:41:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_5 2024-04-23T21:41:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:52.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:52.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:52.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:52.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:52.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:52.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:52.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_5 2024-04-23T21:41:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:52.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:52.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:52.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:52.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:52.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:52.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:52.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:52.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:52.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_5'] 2024-04-23T21:41:52.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:41:52.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_5 2024-04-23T21:41:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:52.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:52.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:52.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:52.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:52.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:52.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:52.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:52.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:52.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:53.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:53.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:53.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:53.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:53.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_5'] 2024-04-23T21:41:53.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:41:53.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:53.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:53.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_5 2024-04-23T21:41:53.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:41:53.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:53.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:53.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:53.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:41:53.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:53.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:53.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:53.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:41:53.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:53.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:53.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:53.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:41:53.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:53.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:53.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:41:53.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:53.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:53.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:53.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:53.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:53.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:53.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:53.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:53.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:53.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:53.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:53.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:53.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:53.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:53.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:53.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:53.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:53.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_5 2024-04-23T21:41:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_5 '' 2024-04-23T21:41:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_5.status 2024-04-23T21:41:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_5.status 2024-04-23T21:41:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:53.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:53.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_5 2024-04-23T21:41:53.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:41:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:41:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=8393], [object_number=4, tag_tid=1, entry_tid=8392], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:53.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:53.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:41:53.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_5 2024-04-23T21:41:53.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:53.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:53.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:53.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_5 '..*' 2024-04-23T21:41:53.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:53.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:53.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:53.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:53.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_5.status 2024-04-23T21:41:53.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_5.status 2024-04-23T21:41:53.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:53.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:53.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_5 2024-04-23T21:41:53.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:53.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:41:53.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:41:53.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:53.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:53.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=8393], [object_number=4, tag_tid=1, entry_tid=8392], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:53.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:53.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:53.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:53.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=8393' '!=' 'object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:53.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:53.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:41:54.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_5 2024-04-23T21:41:54.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:54.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:54.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:41:54.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_5 ']' 2024-04-23T21:41:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_5' 2024-04-23T21:41:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_5 2024-04-23T21:41:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:54.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:54.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:54.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_5 2024-04-23T21:41:54.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:54.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:54.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:54.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:54.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:54.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:54.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:54.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:54.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:54.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:54.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:54.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:54.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_5'] 2024-04-23T21:41:54.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:41:54.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:54.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:54.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_5 2024-04-23T21:41:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:54.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:54.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:54.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:54.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:54.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:54.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:54.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:54.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:54.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:54.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:54.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:54.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:54.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:54.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:54.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:54.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:54.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:54.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:54.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:54.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:54.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:54.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:54.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:54.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:54.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:54.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_5'] 2024-04-23T21:41:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:41:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:54.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:54.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_5 2024-04-23T21:41:54.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:41:54.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:54.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:54.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:41:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:41:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:41:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:41:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:54.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:54.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:54.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:54.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:54.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:54.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:54.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_5 2024-04-23T21:41:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_5 '' 2024-04-23T21:41:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_5.status 2024-04-23T21:41:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_5.status 2024-04-23T21:41:54.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:54.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:54.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_5 2024-04-23T21:41:54.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:54.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:41:54.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:41:54.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:54.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:54.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=8393], [object_number=4, tag_tid=1, entry_tid=8392], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:54.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:54.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:41:54.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_5 2024-04-23T21:41:54.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:54.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:54.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:54.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_5 '..*' 2024-04-23T21:41:54.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:54.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:54.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:54.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:54.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_5.status 2024-04-23T21:41:54.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_5.status 2024-04-23T21:41:54.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:54.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:54.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_5 2024-04-23T21:41:54.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:54.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:41:54.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:41:54.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:54.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:54.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=8393], [object_number=4, tag_tid=1, entry_tid=8392], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:54.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:54.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:54.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:54.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=8393' '!=' 'object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:54.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:54.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:41:56.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_5 2024-04-23T21:41:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:41:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_5 ']' 2024-04-23T21:41:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_5' 2024-04-23T21:41:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_5 2024-04-23T21:41:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:56.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:56.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:56.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:56.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:56.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_5 2024-04-23T21:41:56.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:56.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:56.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:56.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:56.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:56.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:56.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:56.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:56.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:56.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:56.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:56.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:56.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:56.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:56.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:56.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:56.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:56.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:56.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:56.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:56.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:56.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:56.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:56.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:56.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:56.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:56.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:56.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:56.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:56.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:56.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:56.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:56.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:56.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:56.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_5'] 2024-04-23T21:41:56.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:41:56.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_5 2024-04-23T21:41:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:56.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:56.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:56.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:56.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:56.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:56.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:56.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:56.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:56.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:56.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:56.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:56.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:56.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:56.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_5'] 2024-04-23T21:41:56.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:41:56.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:56.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:56.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_5 2024-04-23T21:41:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:41:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:41:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:41:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:56.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:41:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:41:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:56.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:56.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:56.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:56.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:56.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:56.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:56.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:56.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_5 2024-04-23T21:41:56.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:56.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:56.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:56.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_5 '' 2024-04-23T21:41:56.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:56.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:56.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:56.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:56.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_5.status 2024-04-23T21:41:56.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_5.status 2024-04-23T21:41:56.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:56.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:56.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_5 2024-04-23T21:41:56.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:56.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:41:56.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:41:56.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:56.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:56.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=8393], [object_number=4, tag_tid=1, entry_tid=8392], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:56.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:56.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:41:56.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_5 2024-04-23T21:41:56.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:56.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:56.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:56.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_5 '..*' 2024-04-23T21:41:56.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:56.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:56.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:56.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:56.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_5.status 2024-04-23T21:41:56.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_5.status 2024-04-23T21:41:56.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:56.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:56.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_5 2024-04-23T21:41:56.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:56.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:41:56.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:41:56.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:56.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:56.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=8393], [object_number=4, tag_tid=1, entry_tid=8392], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:56.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:56.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:56.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:56.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=8393' '!=' 'object_number=5, tag_tid=1, entry_tid=8393' 2024-04-23T21:41:56.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:41:56.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:41:58.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_5 2024-04-23T21:41:58.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:41:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:41:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_5 ']' 2024-04-23T21:41:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_5' 2024-04-23T21:41:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_5 2024-04-23T21:41:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:41:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:41:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:41:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:41:58.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:41:58.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:41:58.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:41:58.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:58.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_5 2024-04-23T21:41:58.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:41:58.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:58.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:58.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:41:58.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:41:58.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:58.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:58.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:58.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:41:58.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:58.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:58.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:41:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:41:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:41:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:41:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:41:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:41:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:41:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:58.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:58.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:41:58.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:41:58.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:41:58.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:58.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:58.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:41:58.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:58.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_5'] 2024-04-23T21:41:58.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:41:58.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:58.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:58.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_5 2024-04-23T21:41:58.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:41:58.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:58.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:58.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:41:58.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:41:58.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:58.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:58.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:58.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:41:58.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:58.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:58.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:58.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:41:58.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:58.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:41:58.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:41:58.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:58.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:58.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:41:58.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:41:58.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:58.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:58.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:58.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:41:58.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:41:58.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:58.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:58.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:41:58.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:41:58.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:41:58.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:58.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:58.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:41:58.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:58.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_5'] 2024-04-23T21:41:58.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:41:58.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:41:58.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:41:58.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_5 2024-04-23T21:41:58.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:41:58.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:41:58.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:41:58.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:41:58.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:41:58.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:41:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:41:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:41:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:41:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:41:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:41:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:41:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:41:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:41:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:41:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:41:58.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:41:58.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:41:58.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:41:58.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:41:58.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:41:58.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:41:58.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:41:58.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:41:58.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:41:58.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:41:58.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:41:58.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:58.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:58.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:41:58.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_5 2024-04-23T21:41:58.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:58.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_5 2024-04-23T21:41:58.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:58.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:58.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:58.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_5 '' 2024-04-23T21:41:58.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:58.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:58.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:58.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:41:58.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_5.status 2024-04-23T21:41:58.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_5.status 2024-04-23T21:41:58.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:58.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:58.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_5 2024-04-23T21:41:58.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:58.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:41:58.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:41:58.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:58.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:58.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:58.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:58.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:41:58.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_5 2024-04-23T21:41:58.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:58.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:58.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:58.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_5 '..*' 2024-04-23T21:41:58.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:41:58.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:41:58.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:41:58.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:41:58.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_5.status 2024-04-23T21:41:58.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_5.status 2024-04-23T21:41:58.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:41:58.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:58.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_5 2024-04-23T21:41:58.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:58.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:41:58.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:41:58.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:41:58.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:58.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8391], [object_number=6, tag_tid=1, entry_tid=8390]]], state=connected] 2024-04-23T21:41:58.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.status 2024-04-23T21:41:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:41:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:41:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_5 snap 2024-04-23T21:41:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:41:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:41:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:41:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:41:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:41:58.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:41:59.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_5@snap 2024-04-23T21:41:59.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_5': 2024-04-23T21:41:59.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:41:59.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:41:59.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:41:59.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10b5cc8b915 2024-04-23T21:41:59.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10b5cc8b915 2024-04-23T21:41:59.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:41:59.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:41:59.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:41:59.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:41:59.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:32:40 2024 2024-04-23T21:41:59.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:32:40 2024 2024-04-23T21:41:59.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:32:40 2024 2024-04-23T21:41:59.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:41:59.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10b5cc8b915 2024-04-23T21:41:59.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:41:59.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:41:59.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: f0ae1729-5015-402e-a870-d82f77ebb27b 2024-04-23T21:41:59.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:41:59.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:41:59.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_5 snap 2024-04-23T21:41:59.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:41:59.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:41:59.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:41:59.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:41:59.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.export 2024-04-23T21:41:59.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_5.export 2024-04-23T21:41:59.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_5.export 2024-04-23T21:41:59.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_5@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.export 2024-04-23T21:42:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:42:00.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_5@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_5.export 2024-04-23T21:42:00.315 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:42:00.315+0000 7f32a4538640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 1 2024-04-23T21:42:00.315 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:42:00.316+0000 7f32a3d37640 -1 bdev(0x5617e9d36e00 /var/lib/ceph/osd/cluster1-1/block) aio_submit retries 2 2024-04-23T21:42:00.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:42:00.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_5.export 2024-04-23T21:42:00.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_5.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_5.export 2024-04-23T21:42:00.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:00.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:42:00.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_6 2024-04-23T21:42:00.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_6 snap 2024-04-23T21:42:00.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:00.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:00.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:42:00.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:00.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_6@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:42:00.954 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:42:00.954+0000 7f83e9b55640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 1 2024-04-23T21:42:02.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_6 2024-04-23T21:42:02.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:02.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:02.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:42:02.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_6 started 2024-04-23T21:42:02.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:02.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:02.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:42:02.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:42:02.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:42:02.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:02.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:42:03.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_6 started 2024-04-23T21:42:03.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:03.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:03.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:42:03.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:42:03.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:42:03.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:42:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_6 2024-04-23T21:42:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:42:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:42:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:42:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:42:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:03.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:42:03.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:03.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:42:03.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:03.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_6 2024-04-23T21:42:03.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:42:03.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:03.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:03.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:03.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:03.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:03.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:03.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:03.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:03.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:03.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:03.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:03.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:03.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:03.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:42:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:42:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:42:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:42:03.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:03.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:03.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:03.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:03.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_6 2024-04-23T21:42:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_6'] 2024-04-23T21:42:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:42:04.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_6 2024-04-23T21:42:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:42:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:04.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:42:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:42:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:04.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:04.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:04.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:04.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:04.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:42:04.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:42:04.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:04.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:04.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:04.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:04.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_6 2024-04-23T21:42:04.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_6'] 2024-04-23T21:42:04.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:42:04.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:04.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:04.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_6 2024-04-23T21:42:04.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:42:04.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:04.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:04.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:04.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:04.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:04.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:04.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:04.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:04.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:04.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:04.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:04.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:04.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:04.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:04.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:42:04.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:04.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:04.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:04.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:42:04.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:04.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:04.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:04.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:04.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:04.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:04.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:04.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:42:04.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:42:04.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:04.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:04.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:04.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:04.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_6 2024-04-23T21:42:04.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_6'] 2024-04-23T21:42:04.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:42:04.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_6 2024-04-23T21:42:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:42:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:04.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:04.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:04.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:04.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:04.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:04.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:04.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:04.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:04.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:04.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:04.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:42:04.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:04.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:04.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:04.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:42:04.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:42:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:42:04.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:04.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_6 2024-04-23T21:42:04.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:42:04.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:42:04.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:42:04.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:42:04.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:42:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:42:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:42:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_6 2024-04-23T21:42:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:42:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:04.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:42:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:42:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_6 2024-04-23T21:42:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:42:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:42:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:42:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:42:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:42:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:42:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_6 2024-04-23T21:42:04.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:42:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_6 ']' 2024-04-23T21:42:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_6' 2024-04-23T21:42:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_6 2024-04-23T21:42:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:04.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:04.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:04.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:04.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:04.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_6 2024-04-23T21:42:04.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:04.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:04.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:04.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:04.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:04.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:04.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:04.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:04.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:04.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:04.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:04.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:04.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:04.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:04.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:04.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:04.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:04.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:04.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:04.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:04.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:04.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:04.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:04.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:04.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:04.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:04.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:04.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:04.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:04.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:04.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:04.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:04.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:04.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_6 2024-04-23T21:42:04.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_6'] 2024-04-23T21:42:04.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:04.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:04.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:04.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_6 2024-04-23T21:42:04.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:04.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:04.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:04.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:04.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:04.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:04.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:04.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:04.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:04.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:04.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:04.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:04.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:04.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:04.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:04.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:04.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:04.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:04.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:04.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_6 2024-04-23T21:42:04.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_6'] 2024-04-23T21:42:04.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:04.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:04.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:04.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_6 2024-04-23T21:42:04.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:42:04.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:04.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:04.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:04.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:42:04.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:04.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:04.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:04.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:42:04.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:04.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:04.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:04.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:42:04.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:04.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:04.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:42:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:04.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:04.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:04.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:04.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:04.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:04.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:04.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:04.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_6 2024-04-23T21:42:04.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_6'] 2024-04-23T21:42:04.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:04.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:04.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:04.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_6 2024-04-23T21:42:04.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:42:04.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:04.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:04.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:04.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:42:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:42:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:42:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:42:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:04.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:04.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:04.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:04.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:04.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:04.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:04.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:04.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:04.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:04.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:04.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:04.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:04.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:04.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:04.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:04.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_6 2024-04-23T21:42:05.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:05.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_6 2024-04-23T21:42:05.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:05.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:05.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_6 2024-04-23T21:42:05.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_6 '' 2024-04-23T21:42:05.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:05.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:05.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_6 2024-04-23T21:42:05.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:05.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_6.status 2024-04-23T21:42:05.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_6.status 2024-04-23T21:42:05.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:05.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_6.status 2024-04-23T21:42:05.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_6 2024-04-23T21:42:05.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_6.status 2024-04-23T21:42:05.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:42:05.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:42:05.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:05.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11379], [object_number=10, tag_tid=1, entry_tid=11378]]], state=connected] 2024-04-23T21:42:05.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11379], [object_number=10, tag_tid=1, entry_tid=11378]]], state=connected] 2024-04-23T21:42:05.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_6.status 2024-04-23T21:42:05.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:42:05.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_6 2024-04-23T21:42:05.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:05.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:05.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_6 2024-04-23T21:42:05.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_6 '..*' 2024-04-23T21:42:05.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:05.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:05.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_6 2024-04-23T21:42:05.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:05.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_6.status 2024-04-23T21:42:05.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_6.status 2024-04-23T21:42:05.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:05.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_6.status 2024-04-23T21:42:05.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_6 2024-04-23T21:42:05.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_6.status 2024-04-23T21:42:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:42:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:42:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:05.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11379], [object_number=10, tag_tid=1, entry_tid=11378]]], state=connected] 2024-04-23T21:42:05.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11379], [object_number=10, tag_tid=1, entry_tid=11378]]], state=connected] 2024-04-23T21:42:05.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_6.status 2024-04-23T21:42:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:42:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:42:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_6 snap 2024-04-23T21:42:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:42:05.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:42:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:42:05.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:42:06.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_6@snap 2024-04-23T21:42:06.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_6': 2024-04-23T21:42:06.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:42:06.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:42:06.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:42:06.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 116e95c910d6 2024-04-23T21:42:06.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.116e95c910d6 2024-04-23T21:42:06.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:42:06.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:42:06.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:42:06.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:42:06.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:32:48 2024 2024-04-23T21:42:06.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:32:48 2024 2024-04-23T21:42:06.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:33:49 2024 2024-04-23T21:42:06.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:42:06.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 116e95c910d6 2024-04-23T21:42:06.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:42:06.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:42:06.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 943381b0-043f-49a7-8503-0581f0da0469 2024-04-23T21:42:06.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:42:06.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:06.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_6 snap 2024-04-23T21:42:06.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:06.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:42:06.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:06.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:42:06.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_6.export 2024-04-23T21:42:06.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_6.export 2024-04-23T21:42:06.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_6.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_6.export 2024-04-23T21:42:06.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_6@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_6.export 2024-04-23T21:42:06.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:42:06.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_6@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_6.export 2024-04-23T21:42:06.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:42:06.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_6.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_6.export 2024-04-23T21:42:06.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_6.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_6.export 2024-04-23T21:42:06.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:06.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:42:06.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_7 2024-04-23T21:42:06.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_7 snap 2024-04-23T21:42:06.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:06.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:06.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:42:06.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:06.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_7@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:42:07.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_7 2024-04-23T21:42:07.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:07.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:07.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:42:07.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_7 started 2024-04-23T21:42:07.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:07.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:07.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:42:07.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:42:07.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:42:07.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:07.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:42:08.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_7 started 2024-04-23T21:42:08.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:08.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:08.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:42:08.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:42:08.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:42:08.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:42:08.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_7 2024-04-23T21:42:08.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:42:08.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:42:08.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:08.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:08.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:42:08.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:42:08.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:08.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:42:08.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:08.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:42:08.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:08.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_7 2024-04-23T21:42:08.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:42:08.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:42:08.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:08.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:08.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:08.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:42:08.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:08.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:08.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:08.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:08.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:08.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:08.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:08.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:42:08.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:42:08.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:08.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:08.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:08.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:08.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_7 2024-04-23T21:42:09.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:42:09.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:42:09.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:42:09.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:42:09.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:42:09.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:42:09.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:42:09.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:09.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_7 2024-04-23T21:42:09.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:42:09.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:09.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:09.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:42:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:42:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_7 2024-04-23T21:42:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:42:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:42:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:42:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:42:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:42:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:09.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:42:09.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_7 2024-04-23T21:42:09.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:09.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:09.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:42:09.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:09.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_7 ']' 2024-04-23T21:42:09.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_7' 2024-04-23T21:42:09.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_7 2024-04-23T21:42:09.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:09.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:09.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:09.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:09.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:09.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:09.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:09.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_7 2024-04-23T21:42:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:09.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:09.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:09.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:09.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:09.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:09.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:09.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:09.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:09.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:09.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:09.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_7 2024-04-23T21:42:09.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:09.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_7 2024-04-23T21:42:09.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:09.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:09.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:09.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_7 '' 2024-04-23T21:42:09.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:09.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:09.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:09.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:09.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_7.status 2024-04-23T21:42:09.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_7.status 2024-04-23T21:42:09.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:09.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:09.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_7 2024-04-23T21:42:09.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:09.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:42:09.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 4 2024-04-23T21:42:09.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:09.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=17, tag_tid=2, entry_tid=1], [object_number=16, tag_tid=2, entry_tid=0], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194]]], state=connected] 2024-04-23T21:42:09.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=16, tag_tid=1, entry_tid=18196], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194], [object_number=17, tag_tid=1, entry_tid=18193]]], state=connected] 2024-04-23T21:42:09.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:09.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=17, tag_tid=2, entry_tid=1' 2024-04-23T21:42:09.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_7 2024-04-23T21:42:09.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:09.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:09.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:09.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_7 '..*' 2024-04-23T21:42:09.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:09.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:09.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:09.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:09.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_7.status 2024-04-23T21:42:09.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_7.status 2024-04-23T21:42:09.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:09.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:09.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_7 2024-04-23T21:42:09.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:09.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:42:09.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 4 2024-04-23T21:42:09.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:09.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=17, tag_tid=2, entry_tid=1], [object_number=16, tag_tid=2, entry_tid=0], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194]]], state=connected] 2024-04-23T21:42:09.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=16, tag_tid=1, entry_tid=18196], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194], [object_number=17, tag_tid=1, entry_tid=18193]]], state=connected] 2024-04-23T21:42:09.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:09.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=16, tag_tid=1, entry_tid=18196' 2024-04-23T21:42:09.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=17, tag_tid=2, entry_tid=1' -a 'object_number=17, tag_tid=2, entry_tid=1' = 'object_number=16, tag_tid=1, entry_tid=18196' 2024-04-23T21:42:09.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=16, tag_tid=1, entry_tid=18196' '!=' '' 2024-04-23T21:42:09.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:42:09.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=16, tag_tid=1, entry_tid=18196' = '' 2024-04-23T21:42:09.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=16, tag_tid=1, entry_tid=18196' 2024-04-23T21:42:09.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=17, tag_tid=2, entry_tid=1' 2024-04-23T21:42:09.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:42:09.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:09.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:42:09.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=16, tag_tid=1, entry_tid=18196' 2024-04-23T21:42:09.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:42:09.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:09.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:42:09.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=17, tag_tid=2, entry_tid=1' 2024-04-23T21:42:09.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:42:09.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:09.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:42:09.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=16, tag_tid=1, entry_tid=18196' 2024-04-23T21:42:09.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:42:09.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:09.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=18196 2024-04-23T21:42:09.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 18196 2024-04-23T21:42:09.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:42:09.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:09.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:42:09.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_7 2024-04-23T21:42:09.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:09.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:09.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:42:09.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:09.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_7 ']' 2024-04-23T21:42:09.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_7' 2024-04-23T21:42:09.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_7 2024-04-23T21:42:09.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:09.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:09.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:09.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:09.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:09.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:09.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:09.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:09.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:09.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:09.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_7 2024-04-23T21:42:09.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:09.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:09.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:09.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:09.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:09.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:09.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:09.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:09.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:09.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:09.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:09.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:09.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:09.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:09.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:09.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:09.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:09.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:09.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:09.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:09.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_7 2024-04-23T21:42:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_7 2024-04-23T21:42:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_7 '' 2024-04-23T21:42:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:09.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:09.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_7.status 2024-04-23T21:42:09.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_7.status 2024-04-23T21:42:09.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:09.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:09.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_7 2024-04-23T21:42:09.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:09.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:42:09.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 4 2024-04-23T21:42:09.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:09.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=17, tag_tid=2, entry_tid=1], [object_number=16, tag_tid=2, entry_tid=0], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194]]], state=connected] 2024-04-23T21:42:09.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=16, tag_tid=1, entry_tid=18196], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194], [object_number=17, tag_tid=1, entry_tid=18193]]], state=connected] 2024-04-23T21:42:09.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:09.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=17, tag_tid=2, entry_tid=1' 2024-04-23T21:42:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_7 2024-04-23T21:42:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_7 '..*' 2024-04-23T21:42:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:09.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_7.status 2024-04-23T21:42:09.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_7.status 2024-04-23T21:42:09.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:09.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:09.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_7 2024-04-23T21:42:09.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:09.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:42:09.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 4 2024-04-23T21:42:09.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:09.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=17, tag_tid=2, entry_tid=1], [object_number=16, tag_tid=2, entry_tid=0], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194]]], state=connected] 2024-04-23T21:42:09.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=16, tag_tid=1, entry_tid=18196], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194], [object_number=17, tag_tid=1, entry_tid=18193]]], state=connected] 2024-04-23T21:42:09.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:09.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=16, tag_tid=1, entry_tid=18196' 2024-04-23T21:42:09.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=17, tag_tid=2, entry_tid=1' -a 'object_number=17, tag_tid=2, entry_tid=1' = 'object_number=16, tag_tid=1, entry_tid=18196' 2024-04-23T21:42:09.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=16, tag_tid=1, entry_tid=18196' '!=' 'object_number=16, tag_tid=1, entry_tid=18196' 2024-04-23T21:42:09.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:09.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:42:10.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_7 2024-04-23T21:42:10.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:10.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:10.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:42:10.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:10.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_7 ']' 2024-04-23T21:42:10.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_7' 2024-04-23T21:42:10.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_7 2024-04-23T21:42:10.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:10.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:10.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:10.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:10.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:10.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:10.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:10.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:10.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:10.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:10.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_7 2024-04-23T21:42:10.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:10.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:10.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:10.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:10.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:10.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:10.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:10.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:10.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:10.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:10.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:10.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:10.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:10.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:10.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:10.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:10.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:10.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:10.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:10.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:10.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:10.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:10.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:10.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:10.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:10.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:10.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:10.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:10.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:10.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:10.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:10.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:10.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:10.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_7 2024-04-23T21:42:10.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:10.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_7 2024-04-23T21:42:10.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:10.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:10.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:10.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_7 '' 2024-04-23T21:42:10.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:10.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:10.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_7.status 2024-04-23T21:42:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_7.status 2024-04-23T21:42:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:10.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_7 2024-04-23T21:42:10.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:10.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:42:10.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 4 2024-04-23T21:42:10.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:10.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=17, tag_tid=2, entry_tid=1], [object_number=16, tag_tid=2, entry_tid=0], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194]]], state=connected] 2024-04-23T21:42:10.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=16, tag_tid=1, entry_tid=18196], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194], [object_number=17, tag_tid=1, entry_tid=18193]]], state=connected] 2024-04-23T21:42:10.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:10.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=17, tag_tid=2, entry_tid=1' 2024-04-23T21:42:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_7 2024-04-23T21:42:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_7 '..*' 2024-04-23T21:42:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_7.status 2024-04-23T21:42:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_7.status 2024-04-23T21:42:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:10.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:10.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_7 2024-04-23T21:42:10.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:10.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:42:10.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 4 2024-04-23T21:42:10.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:10.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=17, tag_tid=2, entry_tid=1], [object_number=16, tag_tid=2, entry_tid=0], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194]]], state=connected] 2024-04-23T21:42:10.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=16, tag_tid=1, entry_tid=18196], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194], [object_number=17, tag_tid=1, entry_tid=18193]]], state=connected] 2024-04-23T21:42:10.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:10.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=16, tag_tid=1, entry_tid=18196' 2024-04-23T21:42:10.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=17, tag_tid=2, entry_tid=1' -a 'object_number=17, tag_tid=2, entry_tid=1' = 'object_number=16, tag_tid=1, entry_tid=18196' 2024-04-23T21:42:10.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=16, tag_tid=1, entry_tid=18196' '!=' 'object_number=16, tag_tid=1, entry_tid=18196' 2024-04-23T21:42:10.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:10.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:42:11.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_7 2024-04-23T21:42:11.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:11.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:11.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:42:11.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:11.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_7 ']' 2024-04-23T21:42:11.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_7' 2024-04-23T21:42:11.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_7 2024-04-23T21:42:11.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:11.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:11.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:11.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:11.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:11.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:11.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:11.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:11.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:11.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:11.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_7 2024-04-23T21:42:11.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:11.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:11.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:11.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:11.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:11.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:11.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:11.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:11.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:11.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:11.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:11.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:11.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:11.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:11.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:11.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:11.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:11.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:11.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:11.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:11.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_7 2024-04-23T21:42:11.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_7 2024-04-23T21:42:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_7 '' 2024-04-23T21:42:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_7.status 2024-04-23T21:42:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_7.status 2024-04-23T21:42:11.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:11.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:11.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_7 2024-04-23T21:42:11.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:11.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:42:11.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 4 2024-04-23T21:42:11.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:11.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=17, tag_tid=2, entry_tid=1], [object_number=16, tag_tid=2, entry_tid=0], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194]]], state=connected] 2024-04-23T21:42:11.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=16, tag_tid=1, entry_tid=18196], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194], [object_number=17, tag_tid=1, entry_tid=18193]]], state=connected] 2024-04-23T21:42:11.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:11.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=17, tag_tid=2, entry_tid=1' 2024-04-23T21:42:11.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_7 2024-04-23T21:42:11.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:11.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:11.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:11.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_7 '..*' 2024-04-23T21:42:11.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:11.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:11.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:11.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:11.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_7.status 2024-04-23T21:42:11.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_7.status 2024-04-23T21:42:11.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:11.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:11.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_7 2024-04-23T21:42:11.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:11.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:42:11.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 4 2024-04-23T21:42:11.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:11.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=17, tag_tid=2, entry_tid=1], [object_number=16, tag_tid=2, entry_tid=0], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194]]], state=connected] 2024-04-23T21:42:11.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=16, tag_tid=1, entry_tid=18196], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194], [object_number=17, tag_tid=1, entry_tid=18193]]], state=connected] 2024-04-23T21:42:11.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:11.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=16, tag_tid=1, entry_tid=18196' 2024-04-23T21:42:11.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=17, tag_tid=2, entry_tid=1' -a 'object_number=17, tag_tid=2, entry_tid=1' = 'object_number=16, tag_tid=1, entry_tid=18196' 2024-04-23T21:42:11.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=16, tag_tid=1, entry_tid=18196' '!=' 'object_number=16, tag_tid=1, entry_tid=18196' 2024-04-23T21:42:11.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:11.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:42:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_7 2024-04-23T21:42:13.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:42:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_7 ']' 2024-04-23T21:42:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_7' 2024-04-23T21:42:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_7 2024-04-23T21:42:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:13.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:13.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_7 2024-04-23T21:42:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:13.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:13.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:13.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:13.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:13.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:13.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_7 2024-04-23T21:42:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_7 2024-04-23T21:42:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_7 '' 2024-04-23T21:42:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:13.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:13.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_7.status 2024-04-23T21:42:13.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_7.status 2024-04-23T21:42:13.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_7 2024-04-23T21:42:13.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:13.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:42:13.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 4 2024-04-23T21:42:13.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:13.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=17, tag_tid=2, entry_tid=1], [object_number=16, tag_tid=2, entry_tid=0], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194]]], state=connected] 2024-04-23T21:42:13.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=17, tag_tid=2, entry_tid=1], [object_number=16, tag_tid=2, entry_tid=0], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194]]], state=connected] 2024-04-23T21:42:13.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:13.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=17, tag_tid=2, entry_tid=1' 2024-04-23T21:42:13.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_7 2024-04-23T21:42:13.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:13.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:13.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:13.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_7 '..*' 2024-04-23T21:42:13.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:13.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:13.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:42:13.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:13.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_7.status 2024-04-23T21:42:13.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_7.status 2024-04-23T21:42:13.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:13.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:13.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_7 2024-04-23T21:42:13.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:13.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 4 2024-04-23T21:42:13.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 4 2024-04-23T21:42:13.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:13.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=17, tag_tid=2, entry_tid=1], [object_number=16, tag_tid=2, entry_tid=0], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194]]], state=connected] 2024-04-23T21:42:13.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=17, tag_tid=2, entry_tid=1], [object_number=16, tag_tid=2, entry_tid=0], [object_number=19, tag_tid=1, entry_tid=18195], [object_number=18, tag_tid=1, entry_tid=18194]]], state=connected] 2024-04-23T21:42:13.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.status 2024-04-23T21:42:13.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=17, tag_tid=2, entry_tid=1' 2024-04-23T21:42:13.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=17, tag_tid=2, entry_tid=1' -a 'object_number=17, tag_tid=2, entry_tid=1' = 'object_number=17, tag_tid=2, entry_tid=1' 2024-04-23T21:42:13.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:13.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_7 snap 2024-04-23T21:42:13.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:13.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:13.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:42:13.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:13.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:42:13.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:42:13.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:42:14.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_7@snap 2024-04-23T21:42:14.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_7': 2024-04-23T21:42:14.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:42:14.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:42:14.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:42:14.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 11924d3b28e1 2024-04-23T21:42:14.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.11924d3b28e1 2024-04-23T21:42:14.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:42:14.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:42:14.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:42:14.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:42:14.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:32:58 2024 2024-04-23T21:42:14.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:32:58 2024 2024-04-23T21:42:14.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:35:03 2024 2024-04-23T21:42:14.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:42:14.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 11924d3b28e1 2024-04-23T21:42:14.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:42:14.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:42:14.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: a231b240-f37e-4b95-b82b-1f723b71d4f7 2024-04-23T21:42:14.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:42:14.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:14.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_7 snap 2024-04-23T21:42:14.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:14.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:42:14.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:14.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:42:14.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.export 2024-04-23T21:42:14.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_7.export 2024-04-23T21:42:14.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_7.export 2024-04-23T21:42:14.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_7@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.export 2024-04-23T21:42:14.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:42:14.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_7@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_7.export 2024-04-23T21:42:14.714 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:42:14.715+0000 7fc3fb899640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 3 2024-04-23T21:42:14.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:42:14.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_7.export 2024-04-23T21:42:14.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_7.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_7.export 2024-04-23T21:42:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:42:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_8 2024-04-23T21:42:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_8 snap 2024-04-23T21:42:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:42:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:14.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_8@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:42:15.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_8 2024-04-23T21:42:15.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:15.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:15.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:42:15.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_8 started 2024-04-23T21:42:15.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:15.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:15.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:42:15.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:42:15.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:42:15.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:15.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:42:16.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_8 started 2024-04-23T21:42:16.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:16.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:16.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:42:16.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:42:16.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:42:16.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:42:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_8 2024-04-23T21:42:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:42:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:42:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:42:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:42:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:42:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:16.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:42:16.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:16.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_8 2024-04-23T21:42:16.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:42:16.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:16.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:16.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:16.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:16.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:16.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:16.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:16.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:16.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:16.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:16.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:16.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:16.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:16.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:16.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:42:16.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:16.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:16.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:16.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:42:16.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:16.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:16.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:16.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:16.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:16.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:16.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:16.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:42:16.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:42:16.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:17.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:17.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:17.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:17.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_8 2024-04-23T21:42:17.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:17.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:42:17.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:17.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:17.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_8 2024-04-23T21:42:17.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:42:17.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:17.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:17.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:17.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:17.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:17.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:17.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:17.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:17.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:17.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:17.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:17.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:17.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:17.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:17.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:42:17.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:17.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:17.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:17.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:42:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:42:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:42:17.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:17.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:17.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_8 2024-04-23T21:42:17.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:17.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:42:17.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:17.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:17.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_8 2024-04-23T21:42:17.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:42:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:17.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:42:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:42:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:42:17.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:42:17.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:17.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:17.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:17.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:17.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_8 2024-04-23T21:42:17.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:17.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:42:17.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:17.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:17.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_8 2024-04-23T21:42:17.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:42:17.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:17.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:17.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:17.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:42:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:17.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:17.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:17.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:42:17.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:17.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:17.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:17.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:17.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:17.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:17.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:17.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:42:17.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:42:17.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:17.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:17.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:17.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:17.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_8 2024-04-23T21:42:17.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:42:17.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:42:17.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:42:17.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:42:17.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:42:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:42:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:42:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_8 2024-04-23T21:42:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:42:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:17.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:17.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:42:17.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:42:17.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_8 2024-04-23T21:42:17.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:42:17.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:17.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:17.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:42:17.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:42:17.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:42:17.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:42:17.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:17.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:42:17.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_8 2024-04-23T21:42:17.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:17.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:17.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:42:17.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:17.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_8 ']' 2024-04-23T21:42:17.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_8' 2024-04-23T21:42:17.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_8 2024-04-23T21:42:17.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:17.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:17.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:17.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:17.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:17.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:17.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:17.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:17.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:17.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:17.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_8 2024-04-23T21:42:17.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:17.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:17.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:17.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:17.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:17.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:17.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:17.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:17.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:17.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:17.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:17.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:17.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:17.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:17.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:17.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:17.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:17.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:17.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:17.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:17.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:17.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:17.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:17.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:17.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:17.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:17.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:17.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:17.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:17.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:17.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:17.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:17.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:17.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:17.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:17.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:17.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:17.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:17.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_8 2024-04-23T21:42:17.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:17.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:17.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:17.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:17.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:17.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:17.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:17.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:17.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:17.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:17.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:17.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:17.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:17.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:17.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:17.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:17.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:17.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:17.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:17.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:17.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:17.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:17.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:17.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:17.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:17.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:17.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:17.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:17.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:17.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:17.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:17.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:17.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:17.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:17.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:17.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:17.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:17.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:17.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_8 2024-04-23T21:42:17.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:42:17.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:17.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:17.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:17.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:42:17.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:17.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:17.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:42:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:42:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:42:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:17.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:17.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:17.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:17.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:17.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:17.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:17.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:17.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:17.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_8 2024-04-23T21:42:17.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:42:17.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:17.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:42:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:42:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:42:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:42:17.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:17.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:17.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:17.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:17.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:18.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:18.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_8 2024-04-23T21:42:18.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:18.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:18.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:18.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_8 '' 2024-04-23T21:42:18.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_8.status 2024-04-23T21:42:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_8.status 2024-04-23T21:42:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:18.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:18.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_8 2024-04-23T21:42:18.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:18.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:18.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:18.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:18.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:18.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=7749], [object_number=4, tag_tid=1, entry_tid=7748], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:18.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:18.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:18.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_8 2024-04-23T21:42:18.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:18.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:18.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:18.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_8 '..*' 2024-04-23T21:42:18.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:18.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:18.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:18.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:18.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_8.status 2024-04-23T21:42:18.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_8.status 2024-04-23T21:42:18.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:18.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:18.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_8 2024-04-23T21:42:18.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:18.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:18.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:18.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:18.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:18.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=7749], [object_number=4, tag_tid=1, entry_tid=7748], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:18.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:18.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:18.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:18.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=7749' '!=' '' 2024-04-23T21:42:18.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:42:18.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=7749' = '' 2024-04-23T21:42:18.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:18.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:18.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:42:18.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:18.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:42:18.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:18.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:42:18.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:18.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:42:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:42:18.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:18.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:42:18.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:18.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:42:18.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:18.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=7749 2024-04-23T21:42:18.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 7749 2024-04-23T21:42:18.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:42:18.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:18.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:42:18.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_8 2024-04-23T21:42:18.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:18.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:18.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:42:18.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:18.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_8 ']' 2024-04-23T21:42:18.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_8' 2024-04-23T21:42:18.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_8 2024-04-23T21:42:18.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:18.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:18.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:18.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:18.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:18.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:18.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_8 2024-04-23T21:42:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:18.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:18.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:18.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:18.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:18.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:18.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:18.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:18.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:18.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:18.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:18.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:18.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:18.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:18.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:18.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:18.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:18.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:18.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:18.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:18.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_8 2024-04-23T21:42:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:18.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:18.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:18.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:18.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:18.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:18.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:18.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:18.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:18.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:18.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:18.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:18.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:18.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:18.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:18.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:18.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:18.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:18.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:18.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_8 2024-04-23T21:42:18.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:42:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:42:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:42:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:42:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:18.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:18.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:42:18.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:18.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:18.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:18.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:18.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:18.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:18.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:18.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:18.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:18.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:18.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:18.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:18.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:18.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:18.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:18.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:18.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:18.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:18.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:18.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:18.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:18.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_8 2024-04-23T21:42:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:42:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:42:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:42:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:18.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:42:18.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:18.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:18.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:42:18.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:18.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:18.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:18.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:18.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:18.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:18.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:18.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:18.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:18.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:18.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:18.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:18.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:18.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:18.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:18.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:18.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:18.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:18.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:18.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_8 2024-04-23T21:42:18.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:18.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:18.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:18.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_8 '' 2024-04-23T21:42:18.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:18.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:18.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:18.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:18.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_8.status 2024-04-23T21:42:18.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_8.status 2024-04-23T21:42:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_8 2024-04-23T21:42:18.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:19.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:19.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:19.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:19.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:19.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=7749], [object_number=4, tag_tid=1, entry_tid=7748], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:19.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:19.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:19.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_8 2024-04-23T21:42:19.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:19.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:19.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:19.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_8 '..*' 2024-04-23T21:42:19.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:19.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:19.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:19.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:19.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_8.status 2024-04-23T21:42:19.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_8.status 2024-04-23T21:42:19.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:19.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:19.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_8 2024-04-23T21:42:19.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:19.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:19.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:19.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:19.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:19.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=7749], [object_number=4, tag_tid=1, entry_tid=7748], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:19.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:19.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:19.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:19.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=7749' '!=' 'object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:19.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:19.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:42:19.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_8 2024-04-23T21:42:19.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:19.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:19.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:42:19.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:19.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_8 ']' 2024-04-23T21:42:19.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_8' 2024-04-23T21:42:19.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_8 2024-04-23T21:42:19.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:19.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:19.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:19.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:19.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:19.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:19.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:19.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:19.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:19.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:19.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_8 2024-04-23T21:42:19.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:19.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:19.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:19.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:19.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:19.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:19.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:19.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:19.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:19.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:19.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:19.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:19.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:19.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:19.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:19.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:19.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:19.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:19.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:19.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:19.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:19.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:19.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:19.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:19.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:19.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:19.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:19.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:19.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:19.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:19.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:19.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:19.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:19.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:19.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:19.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:19.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:19.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:19.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_8 2024-04-23T21:42:19.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:19.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:19.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:19.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:19.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:19.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:19.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:19.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:19.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:19.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:19.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:19.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:19.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:19.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:19.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:19.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:19.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:19.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:19.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:19.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:19.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:19.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:19.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:19.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:19.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:19.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_8 2024-04-23T21:42:19.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:42:19.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:19.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:19.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:19.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:42:19.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:19.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:19.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:19.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:42:19.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:19.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:19.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:19.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:42:19.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:19.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:19.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:42:19.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:19.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:19.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:19.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:19.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:19.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:19.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:19.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:19.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:19.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:19.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:19.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:19.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:19.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:19.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:19.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:19.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:19.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:19.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:19.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:19.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:19.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:19.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_8 2024-04-23T21:42:19.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:42:19.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:19.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:19.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:19.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:42:19.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:19.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:19.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:19.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:42:19.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:19.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:19.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:19.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:42:19.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:19.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:19.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:42:19.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:19.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:19.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:19.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:19.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:19.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:19.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:19.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:19.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:19.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:19.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:19.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:19.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:19.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:19.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:19.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:19.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:19.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:19.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:19.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_8 2024-04-23T21:42:19.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:19.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:19.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:19.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_8 '' 2024-04-23T21:42:19.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:19.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:19.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:19.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:19.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_8.status 2024-04-23T21:42:19.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_8.status 2024-04-23T21:42:19.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:19.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:19.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_8 2024-04-23T21:42:19.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:20.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:20.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:20.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:20.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:20.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=7749], [object_number=4, tag_tid=1, entry_tid=7748], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:20.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:20.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_8 2024-04-23T21:42:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_8 '..*' 2024-04-23T21:42:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:20.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:20.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:20.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_8.status 2024-04-23T21:42:20.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_8.status 2024-04-23T21:42:20.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:20.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:20.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_8 2024-04-23T21:42:20.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:20.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:20.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:20.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:20.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:20.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=7749], [object_number=4, tag_tid=1, entry_tid=7748], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:20.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:20.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:20.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:20.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=7749' '!=' 'object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:20.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:20.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:42:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_8 2024-04-23T21:42:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:42:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_8 ']' 2024-04-23T21:42:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_8' 2024-04-23T21:42:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_8 2024-04-23T21:42:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:21.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:21.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:21.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:21.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:21.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:21.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:21.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:21.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:21.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:21.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_8 2024-04-23T21:42:21.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:21.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:21.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:21.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:21.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:21.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:21.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:21.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:21.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:21.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:21.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:21.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:21.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:21.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:21.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:21.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:21.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:21.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:21.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:21.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:21.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:21.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:21.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:21.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:21.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_8 2024-04-23T21:42:21.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:21.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:21.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:21.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:21.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:21.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:21.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:21.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:21.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:21.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:21.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:21.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:21.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:21.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:21.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:21.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:21.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:21.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:21.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:21.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:21.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:21.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:21.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:21.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:21.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:21.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:21.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:21.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:21.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:21.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:21.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:21.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:21.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:21.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:21.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:21.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:21.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:21.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:21.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_8 2024-04-23T21:42:21.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:42:21.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:21.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:21.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:42:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:42:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:42:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:42:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:21.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:21.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:21.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:21.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:21.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:21.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:21.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:21.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:21.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:21.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:21.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:21.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:21.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:21.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:21.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:21.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:21.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:21.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:21.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:21.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:21.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_8 2024-04-23T21:42:21.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:42:21.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:21.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:21.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:21.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:42:21.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:21.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:21.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:21.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:42:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:42:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:42:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:21.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:21.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:21.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:21.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:21.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:21.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_8 2024-04-23T21:42:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_8 '' 2024-04-23T21:42:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:21.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_8.status 2024-04-23T21:42:21.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_8.status 2024-04-23T21:42:21.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:21.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:21.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_8 2024-04-23T21:42:21.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:21.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:21.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:21.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:21.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:21.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=7749], [object_number=4, tag_tid=1, entry_tid=7748], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:21.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:21.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:21.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_8 2024-04-23T21:42:21.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:21.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:21.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:21.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_8 '..*' 2024-04-23T21:42:21.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:21.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:21.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_8.status 2024-04-23T21:42:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_8.status 2024-04-23T21:42:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_8 2024-04-23T21:42:21.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:21.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:21.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:21.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:21.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:21.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=7749], [object_number=4, tag_tid=1, entry_tid=7748], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:21.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:21.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:21.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:21.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=7749' '!=' 'object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:21.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:21.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:42:23.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_8 2024-04-23T21:42:23.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:23.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:23.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:42:23.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:23.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_8 ']' 2024-04-23T21:42:23.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_8' 2024-04-23T21:42:23.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_8 2024-04-23T21:42:23.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:23.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:23.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:23.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:23.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:23.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:23.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:23.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:23.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:23.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:23.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_8 2024-04-23T21:42:23.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:23.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:23.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:23.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:23.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:23.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:23.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:23.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:23.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:23.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:23.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:23.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_8 2024-04-23T21:42:23.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:23.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:23.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:23.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:23.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:23.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:23.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:23.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:23.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:23.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:23.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:23.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:23.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:23.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:23.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:23.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:23.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:23.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:23.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:23.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:23.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:23.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:23.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:23.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:23.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:23.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:23.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_8 2024-04-23T21:42:23.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:42:23.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:23.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:23.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:23.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:42:23.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:23.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:23.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:23.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:42:23.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:23.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:23.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:23.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:42:23.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:23.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:23.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:42:23.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:23.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:23.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:23.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:23.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:23.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:23.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:23.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_8 2024-04-23T21:42:23.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:42:23.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:23.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:23.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:23.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:42:23.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:23.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:23.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:23.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:42:23.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:23.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:23.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:23.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:42:23.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:23.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:23.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:42:23.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:23.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:23.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:23.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:23.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:23.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:23.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:23.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:23.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:23.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:23.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:23.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:23.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:23.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:23.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:23.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:23.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:23.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_8 2024-04-23T21:42:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_8 '' 2024-04-23T21:42:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:23.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:23.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:23.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_8.status 2024-04-23T21:42:23.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_8.status 2024-04-23T21:42:23.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:23.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:23.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_8 2024-04-23T21:42:23.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:23.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:23.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:23.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:23.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:23.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=7749], [object_number=4, tag_tid=1, entry_tid=7748], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:23.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:23.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:23.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_8 2024-04-23T21:42:23.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:23.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:23.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:23.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_8 '..*' 2024-04-23T21:42:23.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:23.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:23.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:23.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:23.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_8.status 2024-04-23T21:42:23.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_8.status 2024-04-23T21:42:23.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:23.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:23.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_8 2024-04-23T21:42:23.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:23.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:23.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:23.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:23.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:23.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=7749], [object_number=4, tag_tid=1, entry_tid=7748], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:23.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:23.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:23.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:23.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=7749' '!=' 'object_number=5, tag_tid=1, entry_tid=7749' 2024-04-23T21:42:23.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:23.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:42:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_8 2024-04-23T21:42:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:42:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_8 ']' 2024-04-23T21:42:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_8' 2024-04-23T21:42:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_8 2024-04-23T21:42:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:25.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:25.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:25.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:25.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:25.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:25.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:25.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_8 2024-04-23T21:42:25.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:25.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:25.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:25.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:25.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:25.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:25.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:25.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:25.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:25.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:25.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:25.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:25.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:25.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:25.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:25.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:25.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:25.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:25.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:25.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:25.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:25.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:25.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:25.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:25.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_8 2024-04-23T21:42:25.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:25.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:25.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:25.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:25.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:25.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:25.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:25.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:25.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:25.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:25.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:25.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:25.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:25.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:25.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:25.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:25.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:25.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:25.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:25.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:25.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:25.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:25.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:25.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:25.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:25.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:25.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:25.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:25.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:25.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:25.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_8 2024-04-23T21:42:25.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:42:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:42:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:42:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:42:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:25.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:25.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:42:25.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:25.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:25.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:25.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:25.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:25.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:25.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:25.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:25.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:25.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:25.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:25.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:25.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:25.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:25.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:25.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:25.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:25.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:25.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2024-04-23T21:42:25.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:25.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:25.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:25.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_8 2024-04-23T21:42:25.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:42:25.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:25.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:25.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:25.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:42:25.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:25.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:25.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:25.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:42:25.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:25.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:25.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:25.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:42:25.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:25.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:25.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:42:25.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:25.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:25.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:25.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:25.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:25.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:25.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:25.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:25.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:25.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:25.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:25.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:25.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:25.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:25.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:25.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:25.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:25.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_8 2024-04-23T21:42:26.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:26.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_8 2024-04-23T21:42:26.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:26.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:26.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:26.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_8 '' 2024-04-23T21:42:26.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:26.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:26.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:26.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:26.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_8.status 2024-04-23T21:42:26.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_8.status 2024-04-23T21:42:26.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:26.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:26.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_8 2024-04-23T21:42:26.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:26.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:26.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:26.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:26.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:26.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:26.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:26.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:26.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_8 2024-04-23T21:42:26.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:26.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:26.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:26.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_8 '..*' 2024-04-23T21:42:26.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:26.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:26.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:42:26.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:26.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_8.status 2024-04-23T21:42:26.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_8.status 2024-04-23T21:42:26.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:26.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:26.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_8 2024-04-23T21:42:26.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:26.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:26.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:26.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:26.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:26.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7747], [object_number=6, tag_tid=1, entry_tid=7746]]], state=connected] 2024-04-23T21:42:26.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.status 2024-04-23T21:42:26.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:26.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:26.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:26.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_8 snap 2024-04-23T21:42:26.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:26.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:26.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:42:26.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:26.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:42:26.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:42:26.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:42:27.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_8@snap 2024-04-23T21:42:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_8': 2024-04-23T21:42:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:42:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:42:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:42:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 11b8dacc6120 2024-04-23T21:42:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.11b8dacc6120 2024-04-23T21:42:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:42:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:42:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:42:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:42:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:33:11 2024 2024-04-23T21:42:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:33:11 2024 2024-04-23T21:42:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:33:11 2024 2024-04-23T21:42:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:42:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 11b8dacc6120 2024-04-23T21:42:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:42:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:42:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 91a8999d-21e8-48af-bea2-1a1cefa8f501 2024-04-23T21:42:27.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:42:27.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:27.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_8 snap 2024-04-23T21:42:27.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:27.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:42:27.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:27.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:42:27.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.export 2024-04-23T21:42:27.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_8.export 2024-04-23T21:42:27.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_8.export 2024-04-23T21:42:27.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_8@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.export 2024-04-23T21:42:27.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:42:27.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_8@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_8.export 2024-04-23T21:42:27.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:42:27.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_8.export 2024-04-23T21:42:27.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_8.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_8.export 2024-04-23T21:42:27.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:27.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:42:27.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_9 2024-04-23T21:42:27.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_9 snap 2024-04-23T21:42:27.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:27.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:27.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:42:27.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:27.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_9@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:42:28.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_9 2024-04-23T21:42:28.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:28.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:28.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:42:28.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_9 started 2024-04-23T21:42:28.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:28.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:28.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:42:28.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:42:28.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:42:28.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:28.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:42:29.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_9 started 2024-04-23T21:42:30.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:30.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:30.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:42:30.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:42:30.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:42:30.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:42:30.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_9 2024-04-23T21:42:30.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:42:30.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:42:30.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:30.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:30.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:42:30.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:42:30.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:30.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:42:30.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:30.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:42:30.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:30.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_9 2024-04-23T21:42:30.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:42:30.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:30.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:30.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:30.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:30.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:30.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:30.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:30.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:30.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:30.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:30.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:30.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:30.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:30.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:30.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:42:30.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:30.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:30.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:30.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:42:30.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:30.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:30.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:30.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:30.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:30.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:30.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:30.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:42:30.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:42:30.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:30.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:30.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:30.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:30.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_9 2024-04-23T21:42:30.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_9'] 2024-04-23T21:42:30.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:42:30.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:30.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:30.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_9 2024-04-23T21:42:30.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:42:30.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:30.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:30.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:30.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:42:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:30.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:42:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:42:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:42:30.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:30.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:30.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:30.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:30.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_9 2024-04-23T21:42:30.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_9'] 2024-04-23T21:42:30.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:42:30.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:30.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:30.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_9 2024-04-23T21:42:30.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:42:30.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:30.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:30.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:30.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:30.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:30.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:30.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:30.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:30.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:30.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:30.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:30.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:30.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:30.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:30.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:42:30.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:30.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:30.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:30.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:42:30.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:30.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:30.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:30.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:30.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:30.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:30.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:30.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:42:30.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:42:30.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:30.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:30.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:30.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:30.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_9 2024-04-23T21:42:30.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_9'] 2024-04-23T21:42:30.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:42:30.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:30.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:30.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_9 2024-04-23T21:42:30.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:42:30.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:30.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:30.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:42:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:30.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:30.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:42:30.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:30.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:30.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:30.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:30.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:30.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:30.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:30.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:42:30.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:42:30.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:30.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:30.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:30.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:30.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_9 2024-04-23T21:42:30.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:42:30.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:42:30.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:42:30.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:42:30.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:42:30.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:42:30.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:42:30.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:30.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_9 2024-04-23T21:42:30.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:42:30.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:30.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:30.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:42:30.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:42:30.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_9 2024-04-23T21:42:30.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:42:30.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:30.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:30.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:42:30.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:42:30.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:42:30.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:42:30.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:30.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:42:30.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_9 2024-04-23T21:42:30.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:30.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:30.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:42:30.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:30.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_9 ']' 2024-04-23T21:42:30.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_9' 2024-04-23T21:42:30.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_9 2024-04-23T21:42:30.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:30.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:30.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:30.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:30.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:30.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:30.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:30.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:30.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_9 2024-04-23T21:42:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:30.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:30.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:30.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:30.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:30.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:30.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:30.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:30.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:30.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:30.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:30.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_9 2024-04-23T21:42:30.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_9'] 2024-04-23T21:42:30.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:30.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_9 2024-04-23T21:42:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:30.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:30.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:30.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:30.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:30.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:30.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:30.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:30.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:30.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:30.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:30.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:30.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:30.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:30.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:30.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_9 2024-04-23T21:42:30.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_9'] 2024-04-23T21:42:30.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:30.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:30.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:30.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_9 2024-04-23T21:42:30.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:42:30.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:30.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:30.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:30.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:42:30.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:30.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:30.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:30.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:42:30.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:30.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:30.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:30.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:42:30.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:30.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:30.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:42:30.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:30.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:30.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:30.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:30.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:30.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:30.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:30.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:30.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:30.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:30.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:30.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:30.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:30.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:30.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:30.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:30.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:30.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_9 2024-04-23T21:42:30.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_9'] 2024-04-23T21:42:30.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:30.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:30.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_9 2024-04-23T21:42:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:42:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:42:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:42:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:42:30.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:42:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:30.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:30.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:30.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:30.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:30.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_9 2024-04-23T21:42:31.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:31.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_9 2024-04-23T21:42:31.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:31.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:31.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:42:31.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_9 '' 2024-04-23T21:42:31.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:31.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:31.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:42:31.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:31.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_9.status 2024-04-23T21:42:31.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_9.status 2024-04-23T21:42:31.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:31.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:31.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_9 2024-04-23T21:42:31.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:31.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:42:31.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:42:31.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:31.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15207], [object_number=14, tag_tid=1, entry_tid=15206]]], state=connected] 2024-04-23T21:42:31.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=15207], [object_number=14, tag_tid=1, entry_tid=15206], [object_number=13, tag_tid=1, entry_tid=15205], [object_number=12, tag_tid=1, entry_tid=15204]]], state=connected] 2024-04-23T21:42:31.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:31.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:42:31.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_9 2024-04-23T21:42:31.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:31.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:31.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:42:31.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_9 '..*' 2024-04-23T21:42:31.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:31.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:31.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:42:31.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:31.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_9.status 2024-04-23T21:42:31.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_9.status 2024-04-23T21:42:31.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:31.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:31.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_9 2024-04-23T21:42:31.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:31.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:42:31.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:42:31.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:31.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15207], [object_number=14, tag_tid=1, entry_tid=15206]]], state=connected] 2024-04-23T21:42:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=15207], [object_number=14, tag_tid=1, entry_tid=15206], [object_number=13, tag_tid=1, entry_tid=15205], [object_number=12, tag_tid=1, entry_tid=15204]]], state=connected] 2024-04-23T21:42:31.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:31.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=15, tag_tid=1, entry_tid=15207' 2024-04-23T21:42:31.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=15, tag_tid=1, entry_tid=15207' 2024-04-23T21:42:31.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=15, tag_tid=1, entry_tid=15207' '!=' '' 2024-04-23T21:42:31.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:42:31.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=15, tag_tid=1, entry_tid=15207' = '' 2024-04-23T21:42:31.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=15, tag_tid=1, entry_tid=15207' 2024-04-23T21:42:31.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:42:31.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:42:31.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:31.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:42:31.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=15, tag_tid=1, entry_tid=15207' 2024-04-23T21:42:31.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:42:31.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:31.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:42:31.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:42:31.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:42:31.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:31.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:42:31.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=15, tag_tid=1, entry_tid=15207' 2024-04-23T21:42:31.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:42:31.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=15207 2024-04-23T21:42:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 15207 2024-04-23T21:42:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:42:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:31.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:42:31.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_9 2024-04-23T21:42:31.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:31.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:31.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:42:31.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:31.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_9 ']' 2024-04-23T21:42:31.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_9' 2024-04-23T21:42:31.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_9 2024-04-23T21:42:31.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:31.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:31.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:31.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:31.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:31.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:31.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:31.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:31.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:31.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:31.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_9 2024-04-23T21:42:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:31.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:31.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:31.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:31.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:31.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:31.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_9 2024-04-23T21:42:31.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_9'] 2024-04-23T21:42:31.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:31.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_9 2024-04-23T21:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:31.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:31.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:31.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:31.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:31.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:31.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:31.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:31.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:31.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:31.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:31.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_9 2024-04-23T21:42:31.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_9'] 2024-04-23T21:42:31.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:31.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:31.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:31.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_9 2024-04-23T21:42:31.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:42:31.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:31.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:31.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:31.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:42:31.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:31.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:31.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:31.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:42:31.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:31.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:31.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:31.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:42:31.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:42:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:31.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:31.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:31.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:31.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:31.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_9 2024-04-23T21:42:31.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_9'] 2024-04-23T21:42:31.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:31.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_9 2024-04-23T21:42:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:42:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:42:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:42:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:42:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:42:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:31.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:31.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:31.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:31.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_9 2024-04-23T21:42:31.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:31.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_9 2024-04-23T21:42:31.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:31.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:31.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:42:31.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_9 '' 2024-04-23T21:42:31.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:31.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:31.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:42:31.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:31.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_9.status 2024-04-23T21:42:31.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_9.status 2024-04-23T21:42:31.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:31.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_9 2024-04-23T21:42:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:32.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:42:32.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:42:32.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:32.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15207], [object_number=14, tag_tid=1, entry_tid=15206]]], state=connected] 2024-04-23T21:42:32.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=15207], [object_number=14, tag_tid=1, entry_tid=15206], [object_number=13, tag_tid=1, entry_tid=15205], [object_number=12, tag_tid=1, entry_tid=15204]]], state=connected] 2024-04-23T21:42:32.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:32.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:42:32.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_9 2024-04-23T21:42:32.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:32.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:32.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:42:32.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_9 '..*' 2024-04-23T21:42:32.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:32.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:32.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:42:32.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:32.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_9.status 2024-04-23T21:42:32.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_9.status 2024-04-23T21:42:32.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:32.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_9 2024-04-23T21:42:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:32.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:42:32.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:42:32.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:32.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15207], [object_number=14, tag_tid=1, entry_tid=15206]]], state=connected] 2024-04-23T21:42:32.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=15207], [object_number=14, tag_tid=1, entry_tid=15206], [object_number=13, tag_tid=1, entry_tid=15205], [object_number=12, tag_tid=1, entry_tid=15204]]], state=connected] 2024-04-23T21:42:32.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:32.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=15, tag_tid=1, entry_tid=15207' 2024-04-23T21:42:32.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=15, tag_tid=1, entry_tid=15207' 2024-04-23T21:42:32.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=15, tag_tid=1, entry_tid=15207' '!=' 'object_number=15, tag_tid=1, entry_tid=15207' 2024-04-23T21:42:32.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:32.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:42:32.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_9 2024-04-23T21:42:32.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:32.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:32.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:42:32.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:32.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_9 ']' 2024-04-23T21:42:32.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_9' 2024-04-23T21:42:32.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_9 2024-04-23T21:42:32.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:32.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:32.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:32.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:32.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:32.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:32.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:32.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:32.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:32.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:32.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_9 2024-04-23T21:42:32.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:32.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:32.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:32.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:32.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:32.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:32.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:32.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:32.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:32.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:32.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:32.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:32.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:32.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:32.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:32.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:32.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:32.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:32.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:32.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_9 2024-04-23T21:42:32.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_9'] 2024-04-23T21:42:32.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:32.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_9 2024-04-23T21:42:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:32.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:32.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:32.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:32.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:32.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:32.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:32.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:32.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:32.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:32.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:32.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:32.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:32.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:32.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:32.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:32.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:32.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:32.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_9 2024-04-23T21:42:32.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_9'] 2024-04-23T21:42:32.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:32.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_9 2024-04-23T21:42:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:42:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:42:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:42:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:32.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:42:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:42:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:32.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:32.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:32.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:32.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:32.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:32.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:32.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:32.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_9 2024-04-23T21:42:32.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_9'] 2024-04-23T21:42:32.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:32.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:32.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:32.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_9 2024-04-23T21:42:32.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:42:32.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:32.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:32.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:32.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:42:32.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:42:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:42:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:42:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:32.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:32.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:32.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:32.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:32.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:32.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:32.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:32.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:32.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:32.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:32.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:32.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:32.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:32.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:32.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_9 2024-04-23T21:42:32.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:32.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_9 2024-04-23T21:42:32.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:32.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:32.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:42:32.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_9 '' 2024-04-23T21:42:32.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:32.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:32.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:42:32.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:32.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_9.status 2024-04-23T21:42:32.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_9.status 2024-04-23T21:42:32.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:32.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:32.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_9 2024-04-23T21:42:32.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:32.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:42:32.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:42:32.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:32.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15207], [object_number=14, tag_tid=1, entry_tid=15206]]], state=connected] 2024-04-23T21:42:32.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15207], [object_number=14, tag_tid=1, entry_tid=15206]]], state=connected] 2024-04-23T21:42:32.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:32.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:42:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_9 2024-04-23T21:42:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:42:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_9 '..*' 2024-04-23T21:42:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:42:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:32.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_9.status 2024-04-23T21:42:32.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_9.status 2024-04-23T21:42:32.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:32.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:32.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_9 2024-04-23T21:42:32.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:32.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:42:32.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:42:32.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:32.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15207], [object_number=14, tag_tid=1, entry_tid=15206]]], state=connected] 2024-04-23T21:42:32.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15207], [object_number=14, tag_tid=1, entry_tid=15206]]], state=connected] 2024-04-23T21:42:32.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.status 2024-04-23T21:42:32.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:42:32.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:42:32.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:32.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_9 snap 2024-04-23T21:42:32.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:32.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:32.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:42:32.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:32.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:42:32.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:42:32.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:42:33.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_9@snap 2024-04-23T21:42:34.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_9': 2024-04-23T21:42:34.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:42:34.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:42:34.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:42:34.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10b5f0e86e79 2024-04-23T21:42:34.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10b5f0e86e79 2024-04-23T21:42:34.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:42:34.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:42:34.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:42:34.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:42:34.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:33:17 2024 2024-04-23T21:42:34.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:33:17 2024 2024-04-23T21:42:34.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:34:18 2024 2024-04-23T21:42:34.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:42:34.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10b5f0e86e79 2024-04-23T21:42:34.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:42:34.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:42:34.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 356e9a34-f5a2-4ec4-ba0d-3cae96d470f8 2024-04-23T21:42:34.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:42:34.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:34.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_9 snap 2024-04-23T21:42:34.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:34.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:42:34.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:42:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.export 2024-04-23T21:42:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_9.export 2024-04-23T21:42:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_9.export 2024-04-23T21:42:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_9@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.export 2024-04-23T21:42:34.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:42:34.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_9@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_9.export 2024-04-23T21:42:34.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:42:34.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_9.export 2024-04-23T21:42:34.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_9.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_9.export 2024-04-23T21:42:34.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:34.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:42:34.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_10 2024-04-23T21:42:34.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_10 snap 2024-04-23T21:42:34.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:34.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:34.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:42:34.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:34.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_10@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:42:36.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_10 2024-04-23T21:42:36.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:36.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:36.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:42:36.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_10 started 2024-04-23T21:42:36.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:36.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:36.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:42:36.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:42:36.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:42:36.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:36.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:42:37.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_10 started 2024-04-23T21:42:37.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:37.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:37.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:42:37.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:42:37.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:42:37.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:42:37.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_10 2024-04-23T21:42:37.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:42:37.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:42:37.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:37.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:37.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:42:37.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:42:37.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:37.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:42:37.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:37.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:42:37.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:37.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_10 2024-04-23T21:42:37.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:42:37.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:37.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:37.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:37.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:37.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:37.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:37.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:37.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:37.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:37.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:37.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:37.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:37.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:37.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:37.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:42:37.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:37.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:37.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:37.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:42:37.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:37.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:37.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:37.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:37.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:37.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:37.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:37.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:42:37.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:42:37.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:37.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:37.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:37.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:37.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_10 2024-04-23T21:42:37.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2024-04-23T21:42:37.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:42:37.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:37.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:37.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_10 2024-04-23T21:42:37.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:42:37.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:37.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:37.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:37.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:37.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:37.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:37.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:37.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:37.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:37.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:37.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:37.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:37.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:37.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:37.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:42:37.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:37.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:37.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:37.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:42:37.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:37.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:37.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:37.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:37.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:37.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:37.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:37.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:42:37.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:42:37.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:37.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:37.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:37.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:37.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_10 2024-04-23T21:42:37.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2024-04-23T21:42:37.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:42:37.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_10 2024-04-23T21:42:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:42:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:37.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:37.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:37.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:37.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:42:37.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:37.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:37.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:37.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:42:37.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:37.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:37.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:37.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:37.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:37.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:37.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:37.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:42:37.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:42:37.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:37.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:37.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:37.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:37.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_10 2024-04-23T21:42:37.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2024-04-23T21:42:37.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:42:37.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:37.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:37.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_10 2024-04-23T21:42:37.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:42:37.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:37.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:37.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:37.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:37.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:37.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:37.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:37.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:37.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:37.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:37.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:37.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:37.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:37.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:37.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:42:37.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:37.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:37.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:37.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:42:37.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:37.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:37.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:37.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:37.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:37.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:37.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:37.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:42:37.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:42:37.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:37.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:37.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:37.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:37.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_10 2024-04-23T21:42:37.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:42:37.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:42:37.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:42:37.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:42:37.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:42:37.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:42:37.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:42:37.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:37.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_10 2024-04-23T21:42:37.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:42:37.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:37.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:37.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:42:37.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:42:37.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_10 2024-04-23T21:42:37.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:42:37.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:37.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:37.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:42:37.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:42:37.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:42:37.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:42:37.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:37.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:42:37.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_10 2024-04-23T21:42:37.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:37.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:37.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:42:37.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:37.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_10 ']' 2024-04-23T21:42:37.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_10' 2024-04-23T21:42:37.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_10 2024-04-23T21:42:37.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:37.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:37.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:37.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:37.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:37.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:37.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:37.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:37.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:37.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:37.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_10 2024-04-23T21:42:37.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:37.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:37.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:37.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:37.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:37.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:37.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:37.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:37.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:37.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:37.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:37.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:37.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:37.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:37.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:37.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:37.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:37.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:37.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:37.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:37.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:37.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:37.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:37.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:37.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:37.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:37.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:37.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:37.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:37.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:37.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:37.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:37.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:37.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_10 2024-04-23T21:42:37.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2024-04-23T21:42:37.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:37.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:37.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:37.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_10 2024-04-23T21:42:37.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:37.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:37.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:37.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:37.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:37.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:37.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:37.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:37.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:37.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:37.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:37.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:37.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:37.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:37.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:37.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:37.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:37.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:37.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:37.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_10 2024-04-23T21:42:37.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2024-04-23T21:42:37.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:37.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:37.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:37.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_10 2024-04-23T21:42:37.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:42:37.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:37.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:37.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:37.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:42:37.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:37.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:37.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:37.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:42:37.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:37.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:37.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:37.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:42:37.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:37.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:37.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:42:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:37.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:37.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:37.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:37.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:37.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:37.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_10 2024-04-23T21:42:37.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2024-04-23T21:42:37.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:37.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:37.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:37.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_10 2024-04-23T21:42:37.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:42:37.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:37.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:37.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:37.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:42:37.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:37.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:37.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:37.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:42:37.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:37.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:37.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:37.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:42:37.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:37.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:37.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:42:37.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:37.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:37.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:37.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:37.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:37.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:37.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:37.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:37.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:37.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:37.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:37.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:37.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:37.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:37.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:37.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:37.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:37.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_10 2024-04-23T21:42:38.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:38.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_10 2024-04-23T21:42:38.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:38.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:38.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:42:38.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_10 '' 2024-04-23T21:42:38.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:38.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:38.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:42:38.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:38.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_10.status 2024-04-23T21:42:38.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_10.status 2024-04-23T21:42:38.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:38.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:38.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_10 2024-04-23T21:42:38.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:38.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:38.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:38.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:38.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6227], [object_number=6, tag_tid=1, entry_tid=6226]]], state=connected] 2024-04-23T21:42:38.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=6228], [object_number=7, tag_tid=1, entry_tid=6227], [object_number=6, tag_tid=1, entry_tid=6226], [object_number=5, tag_tid=1, entry_tid=6225]]], state=connected] 2024-04-23T21:42:38.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:38.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:38.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_10 2024-04-23T21:42:38.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:38.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:38.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:42:38.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_10 '..*' 2024-04-23T21:42:38.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:38.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:38.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:42:38.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:38.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_10.status 2024-04-23T21:42:38.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_10.status 2024-04-23T21:42:38.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:38.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:38.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_10 2024-04-23T21:42:38.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:38.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:38.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:38.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:38.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6227], [object_number=6, tag_tid=1, entry_tid=6226]]], state=connected] 2024-04-23T21:42:38.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=6228], [object_number=7, tag_tid=1, entry_tid=6227], [object_number=6, tag_tid=1, entry_tid=6226], [object_number=5, tag_tid=1, entry_tid=6225]]], state=connected] 2024-04-23T21:42:38.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=4, tag_tid=1, entry_tid=6228' 2024-04-23T21:42:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=4, tag_tid=1, entry_tid=6228' 2024-04-23T21:42:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=4, tag_tid=1, entry_tid=6228' '!=' '' 2024-04-23T21:42:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:42:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=4, tag_tid=1, entry_tid=6228' = '' 2024-04-23T21:42:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=4, tag_tid=1, entry_tid=6228' 2024-04-23T21:42:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:42:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:38.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:42:38.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=4, tag_tid=1, entry_tid=6228' 2024-04-23T21:42:38.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:42:38.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:38.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:42:38.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:38.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:42:38.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:38.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:42:38.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=4, tag_tid=1, entry_tid=6228' 2024-04-23T21:42:38.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:42:38.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:38.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=6228 2024-04-23T21:42:38.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 6228 2024-04-23T21:42:38.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:42:38.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:38.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:42:38.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_10 2024-04-23T21:42:38.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:38.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:38.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:42:38.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:38.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_10 ']' 2024-04-23T21:42:38.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_10' 2024-04-23T21:42:38.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_10 2024-04-23T21:42:38.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:38.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:38.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:38.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:38.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:38.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:38.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:38.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:38.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:38.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:38.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_10 2024-04-23T21:42:38.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:38.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:38.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:38.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:38.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:38.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:38.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:38.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:38.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:38.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:38.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:38.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:38.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:38.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:38.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:38.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:38.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:38.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:38.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:38.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:38.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:38.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:38.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:38.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:38.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:38.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:38.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:38.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:38.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:38.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:38.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:38.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:38.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:38.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_10 2024-04-23T21:42:38.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2024-04-23T21:42:38.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:38.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:38.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:38.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_10 2024-04-23T21:42:38.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:38.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:38.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:38.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:38.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:38.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:38.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:38.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:38.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:38.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:38.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:38.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:38.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_10 2024-04-23T21:42:38.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2024-04-23T21:42:38.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:38.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:38.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:38.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_10 2024-04-23T21:42:38.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:42:38.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:38.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:38.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:38.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:42:38.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:38.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:38.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:38.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:42:38.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:38.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:38.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:38.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:42:38.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:42:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:38.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:38.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:38.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:38.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:38.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:38.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_10 2024-04-23T21:42:38.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2024-04-23T21:42:38.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:38.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:38.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:38.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_10 2024-04-23T21:42:38.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:42:38.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:38.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:38.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:38.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:42:38.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:42:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:42:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:42:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:38.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:38.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:38.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:38.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:38.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:38.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:38.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:38.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:38.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:38.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:38.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:38.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_10 2024-04-23T21:42:38.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:38.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_10 2024-04-23T21:42:38.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:38.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:38.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:42:38.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_10 '' 2024-04-23T21:42:38.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:38.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:38.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:42:38.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:38.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_10.status 2024-04-23T21:42:38.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_10.status 2024-04-23T21:42:38.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:38.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:38.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_10 2024-04-23T21:42:38.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:38.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:38.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:38.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:38.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6227], [object_number=6, tag_tid=1, entry_tid=6226]]], state=connected] 2024-04-23T21:42:38.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=6228], [object_number=7, tag_tid=1, entry_tid=6227], [object_number=6, tag_tid=1, entry_tid=6226], [object_number=5, tag_tid=1, entry_tid=6225]]], state=connected] 2024-04-23T21:42:38.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_10 2024-04-23T21:42:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:42:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_10 '..*' 2024-04-23T21:42:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:42:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_10.status 2024-04-23T21:42:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_10.status 2024-04-23T21:42:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:38.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_10 2024-04-23T21:42:38.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:38.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:38.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:38.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:38.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6227], [object_number=6, tag_tid=1, entry_tid=6226]]], state=connected] 2024-04-23T21:42:38.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=6228], [object_number=7, tag_tid=1, entry_tid=6227], [object_number=6, tag_tid=1, entry_tid=6226], [object_number=5, tag_tid=1, entry_tid=6225]]], state=connected] 2024-04-23T21:42:38.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:38.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=4, tag_tid=1, entry_tid=6228' 2024-04-23T21:42:38.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=4, tag_tid=1, entry_tid=6228' 2024-04-23T21:42:38.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=4, tag_tid=1, entry_tid=6228' '!=' 'object_number=4, tag_tid=1, entry_tid=6228' 2024-04-23T21:42:38.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:38.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:42:39.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_10 2024-04-23T21:42:39.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:39.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:39.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:42:39.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:39.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_10 ']' 2024-04-23T21:42:39.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_10' 2024-04-23T21:42:39.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_10 2024-04-23T21:42:39.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:39.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:39.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:39.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:39.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:39.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:39.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:39.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:39.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_10 2024-04-23T21:42:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:39.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:39.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:39.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:39.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:39.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:39.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:39.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:39.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:39.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:39.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:39.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:39.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_10 2024-04-23T21:42:39.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2024-04-23T21:42:39.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:39.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_10 2024-04-23T21:42:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:39.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:39.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:39.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:39.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:39.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:39.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:39.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:39.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:39.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:39.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:39.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:39.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:39.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:39.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_10 2024-04-23T21:42:39.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2024-04-23T21:42:39.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:39.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_10 2024-04-23T21:42:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:42:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:42:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:42:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:42:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:42:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:39.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:39.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:39.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:39.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:39.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:39.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_10 2024-04-23T21:42:39.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2024-04-23T21:42:39.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:39.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:39.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:39.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_10 2024-04-23T21:42:39.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:42:39.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:39.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:39.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:39.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:42:39.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:39.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:39.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:39.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:42:39.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:39.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:39.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:39.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:42:39.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:39.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:39.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:42:39.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:39.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:39.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:39.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:39.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:39.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:39.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:39.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:39.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:39.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:39.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:39.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:39.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:39.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:39.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:39.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:39.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:39.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_10 2024-04-23T21:42:39.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:39.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_10 2024-04-23T21:42:39.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:39.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:39.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:42:39.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_10 '' 2024-04-23T21:42:39.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:39.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:39.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:42:39.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:39.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_10.status 2024-04-23T21:42:39.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_10.status 2024-04-23T21:42:39.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:39.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:39.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_10 2024-04-23T21:42:39.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:39.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:39.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:39.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:39.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6227], [object_number=6, tag_tid=1, entry_tid=6226]]], state=connected] 2024-04-23T21:42:39.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=6228], [object_number=7, tag_tid=1, entry_tid=6227], [object_number=6, tag_tid=1, entry_tid=6226], [object_number=5, tag_tid=1, entry_tid=6225]]], state=connected] 2024-04-23T21:42:39.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:39.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:39.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_10 2024-04-23T21:42:39.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:39.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:39.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:42:39.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_10 '..*' 2024-04-23T21:42:39.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:39.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:39.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:42:39.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:39.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_10.status 2024-04-23T21:42:39.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_10.status 2024-04-23T21:42:39.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:39.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:39.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_10 2024-04-23T21:42:39.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:39.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:39.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:39.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:39.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6227], [object_number=6, tag_tid=1, entry_tid=6226]]], state=connected] 2024-04-23T21:42:39.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=6228], [object_number=7, tag_tid=1, entry_tid=6227], [object_number=6, tag_tid=1, entry_tid=6226], [object_number=5, tag_tid=1, entry_tid=6225]]], state=connected] 2024-04-23T21:42:39.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:39.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=4, tag_tid=1, entry_tid=6228' 2024-04-23T21:42:39.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=4, tag_tid=1, entry_tid=6228' 2024-04-23T21:42:39.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=4, tag_tid=1, entry_tid=6228' '!=' 'object_number=4, tag_tid=1, entry_tid=6228' 2024-04-23T21:42:39.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:39.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:42:40.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_10 2024-04-23T21:42:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:42:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_10 ']' 2024-04-23T21:42:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_10' 2024-04-23T21:42:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_10 2024-04-23T21:42:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:40.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:40.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:40.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_10 2024-04-23T21:42:40.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:40.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:40.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:40.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:40.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:40.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:40.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:40.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:40.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:40.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:40.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:40.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:40.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:40.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:40.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:40.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:40.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:40.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:40.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:40.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:40.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_10 2024-04-23T21:42:40.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2024-04-23T21:42:40.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:40.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:40.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:40.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_10 2024-04-23T21:42:40.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:40.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:40.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:40.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:40.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:40.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:40.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:40.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:40.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:40.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:40.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:40.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:40.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:40.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:40.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:40.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:40.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:40.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:40.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:40.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:40.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:40.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:40.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:40.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:40.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:40.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:40.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:40.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:40.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:40.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:40.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:40.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:40.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:40.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_10 2024-04-23T21:42:41.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2024-04-23T21:42:41.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:41.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:41.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_10 2024-04-23T21:42:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:42:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:42:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:42:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:42:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:41.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:42:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:42:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:42:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:42:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:42:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:42:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:42:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:41.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:41.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:42:41.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:42:41.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:42:41.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:42:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_10 2024-04-23T21:42:41.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2024-04-23T21:42:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_10 2024-04-23T21:42:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:42:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:42:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:42:41.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:42:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:42:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:42:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:42:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:42:41.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:42:41.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:41.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:41.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:41.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:42:41.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:42:41.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:41.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:41.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:42:41.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:42:41.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:42:41.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:41.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:41.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:42:41.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_10 2024-04-23T21:42:41.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:41.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_10 2024-04-23T21:42:41.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:41.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:41.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:42:41.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_10 '' 2024-04-23T21:42:41.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:41.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:41.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:42:41.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:41.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_10.status 2024-04-23T21:42:41.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_10.status 2024-04-23T21:42:41.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:41.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:41.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_10 2024-04-23T21:42:41.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:41.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:41.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:41.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:41.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6227], [object_number=6, tag_tid=1, entry_tid=6226]]], state=connected] 2024-04-23T21:42:41.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6227], [object_number=6, tag_tid=1, entry_tid=6226]]], state=connected] 2024-04-23T21:42:41.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:41.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:41.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_10 2024-04-23T21:42:41.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:41.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:41.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:42:41.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_10 '..*' 2024-04-23T21:42:41.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:41.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:41.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:42:41.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:41.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_10.status 2024-04-23T21:42:41.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_10.status 2024-04-23T21:42:41.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:41.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:41.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_10 2024-04-23T21:42:41.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:41.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:42:41.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:42:41.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:41.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6227], [object_number=6, tag_tid=1, entry_tid=6226]]], state=connected] 2024-04-23T21:42:41.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6227], [object_number=6, tag_tid=1, entry_tid=6226]]], state=connected] 2024-04-23T21:42:41.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.status 2024-04-23T21:42:41.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:41.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:42:41.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:41.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_10 snap 2024-04-23T21:42:41.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:41.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:41.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:42:41.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:41.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:42:41.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:42:41.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:42:42.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_10@snap 2024-04-23T21:42:42.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_10': 2024-04-23T21:42:42.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:42:42.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:42:42.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:42:42.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 116e52f5185b 2024-04-23T21:42:42.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.116e52f5185b 2024-04-23T21:42:42.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:42:42.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:42:42.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:42:42.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:42:42.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:33:31 2024 2024-04-23T21:42:42.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:33:31 2024 2024-04-23T21:42:42.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:33:31 2024 2024-04-23T21:42:42.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:42:42.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 116e52f5185b 2024-04-23T21:42:42.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:42:42.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:42:42.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: a5003cc7-3a00-4b0f-b660-c9391be7ce73 2024-04-23T21:42:42.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:42:42.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:42.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_10 snap 2024-04-23T21:42:42.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:42.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:42:42.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:42.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:42:42.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.export 2024-04-23T21:42:42.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_10.export 2024-04-23T21:42:42.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_10.export 2024-04-23T21:42:42.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_10@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.export 2024-04-23T21:42:42.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:42:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_10@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_10.export 2024-04-23T21:42:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:42:42.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_10.export 2024-04-23T21:42:42.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_10.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_10.export 2024-04-23T21:42:42.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:42.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:42:42.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_11 2024-04-23T21:42:42.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_11 snap 2024-04-23T21:42:42.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:42.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:42.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:42:42.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:42.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_11@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:42:45.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_11 2024-04-23T21:42:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:42:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_11 started 2024-04-23T21:42:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:42:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:42:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:42:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:45.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:42:46.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_11 started 2024-04-23T21:42:46.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:46.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:46.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:42:46.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:42:46.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:42:46.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:42:46.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_11 2024-04-23T21:42:46.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:42:46.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:46.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:42:46.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:46.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:42:46.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:42:46.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:46.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:42:46.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:46.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:42:46.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_11 2024-04-23T21:42:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:42:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:46.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:42:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:42:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:46.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:46.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:42:46.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:42:46.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:46.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:46.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:46.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:46.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_11 2024-04-23T21:42:46.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_11'] 2024-04-23T21:42:46.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:42:46.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_11 2024-04-23T21:42:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:42:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:42:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:42:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:42:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:42:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:46.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:46.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:46.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:46.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_11 2024-04-23T21:42:46.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:42:46.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:42:46.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:42:46.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:42:46.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:42:46.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:42:46.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:42:46.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:46.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_11 2024-04-23T21:42:46.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:42:46.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:46.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:46.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:42:46.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:42:46.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_11 2024-04-23T21:42:46.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:42:46.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:46.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:46.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:42:46.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:42:46.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:42:46.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:42:46.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:46.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:42:46.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_11 2024-04-23T21:42:46.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:46.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:46.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:42:46.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:46.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_11 ']' 2024-04-23T21:42:46.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_11' 2024-04-23T21:42:46.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_11 2024-04-23T21:42:46.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:46.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:46.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:46.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:46.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:46.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:46.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:46.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:46.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:46.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:46.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_11 2024-04-23T21:42:46.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:46.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:46.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:46.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:46.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:46.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:46.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:46.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:46.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:46.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:46.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:46.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:46.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:46.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:46.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:46.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:46.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:46.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:46.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:46.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:46.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:46.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:46.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:46.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:46.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:46.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:46.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:46.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:46.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:46.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:46.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:46.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:46.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:46.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_11 2024-04-23T21:42:46.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_11'] 2024-04-23T21:42:46.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:46.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:46.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:46.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_11 2024-04-23T21:42:46.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:46.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:46.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:46.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:46.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:46.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:46.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:46.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:46.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:46.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:46.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:46.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:46.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:46.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:46.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:46.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:46.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:46.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:46.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:46.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:46.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:46.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:46.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:46.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:46.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:46.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:46.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:46.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:46.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:46.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:46.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:46.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:46.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:46.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_11 2024-04-23T21:42:46.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:46.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_11 2024-04-23T21:42:46.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:46.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:46.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:42:46.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_11 '' 2024-04-23T21:42:46.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:46.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:46.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:42:46.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:46.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_11.status 2024-04-23T21:42:46.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_11.status 2024-04-23T21:42:46.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:46.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:46.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_11 2024-04-23T21:42:46.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:46.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:42:46.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:42:46.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:46.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15055], [object_number=14, tag_tid=1, entry_tid=15054]]], state=connected] 2024-04-23T21:42:46.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=15056], [object_number=15, tag_tid=1, entry_tid=15055], [object_number=14, tag_tid=1, entry_tid=15054], [object_number=13, tag_tid=1, entry_tid=15053]]], state=connected] 2024-04-23T21:42:46.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:46.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:42:46.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_11 2024-04-23T21:42:46.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:46.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:46.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:42:46.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_11 '..*' 2024-04-23T21:42:46.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:46.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:46.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:42:46.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:46.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_11.status 2024-04-23T21:42:46.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_11.status 2024-04-23T21:42:46.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:46.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:46.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_11 2024-04-23T21:42:46.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:46.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:42:46.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:42:46.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:46.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15055], [object_number=14, tag_tid=1, entry_tid=15054]]], state=connected] 2024-04-23T21:42:46.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=15056], [object_number=15, tag_tid=1, entry_tid=15055], [object_number=14, tag_tid=1, entry_tid=15054], [object_number=13, tag_tid=1, entry_tid=15053]]], state=connected] 2024-04-23T21:42:46.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:46.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=12, tag_tid=1, entry_tid=15056' 2024-04-23T21:42:46.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=12, tag_tid=1, entry_tid=15056' 2024-04-23T21:42:46.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=12, tag_tid=1, entry_tid=15056' '!=' '' 2024-04-23T21:42:46.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:42:46.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=12, tag_tid=1, entry_tid=15056' = '' 2024-04-23T21:42:46.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=12, tag_tid=1, entry_tid=15056' 2024-04-23T21:42:46.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:42:46.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:42:46.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:46.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:42:46.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=12, tag_tid=1, entry_tid=15056' 2024-04-23T21:42:46.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:42:46.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:46.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:42:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:42:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:42:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:46.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:42:46.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=12, tag_tid=1, entry_tid=15056' 2024-04-23T21:42:46.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:42:46.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:46.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=15056 2024-04-23T21:42:46.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 15056 2024-04-23T21:42:46.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:42:46.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:46.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:42:46.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_11 2024-04-23T21:42:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:42:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_11 ']' 2024-04-23T21:42:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_11' 2024-04-23T21:42:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_11 2024-04-23T21:42:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:46.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:46.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:46.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:46.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:46.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_11 2024-04-23T21:42:46.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:46.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:46.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:46.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:46.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:46.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:46.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:46.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:46.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:46.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:46.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:46.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:46.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:46.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:46.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:46.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:46.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:46.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:46.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:46.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:46.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:46.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:46.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:46.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:46.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:46.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:46.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:46.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:46.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:46.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:46.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:46.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:46.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:46.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_11 2024-04-23T21:42:47.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_11'] 2024-04-23T21:42:47.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:47.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:47.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:47.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_11 2024-04-23T21:42:47.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:47.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:47.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:47.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:47.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:47.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:47.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:47.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:47.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:47.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:47.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:47.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:47.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:47.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:47.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:47.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:47.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:47.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:47.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:47.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:47.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:47.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:47.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:47.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:47.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:47.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:47.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:47.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:47.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:47.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:47.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:47.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:47.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:47.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_11 2024-04-23T21:42:47.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:47.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_11 2024-04-23T21:42:47.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:47.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:47.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:42:47.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_11 '' 2024-04-23T21:42:47.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:47.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:47.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:42:47.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:47.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_11.status 2024-04-23T21:42:47.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_11.status 2024-04-23T21:42:47.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:47.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:47.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_11 2024-04-23T21:42:47.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:47.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:42:47.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:42:47.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:47.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15055], [object_number=14, tag_tid=1, entry_tid=15054]]], state=connected] 2024-04-23T21:42:47.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=15056], [object_number=15, tag_tid=1, entry_tid=15055], [object_number=14, tag_tid=1, entry_tid=15054], [object_number=13, tag_tid=1, entry_tid=15053]]], state=connected] 2024-04-23T21:42:47.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:47.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:42:47.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_11 2024-04-23T21:42:47.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:47.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:47.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:42:47.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_11 '..*' 2024-04-23T21:42:47.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:47.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:47.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:42:47.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_11.status 2024-04-23T21:42:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_11.status 2024-04-23T21:42:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_11 2024-04-23T21:42:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:47.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:42:47.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:42:47.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:47.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15055], [object_number=14, tag_tid=1, entry_tid=15054]]], state=connected] 2024-04-23T21:42:47.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=15056], [object_number=15, tag_tid=1, entry_tid=15055], [object_number=14, tag_tid=1, entry_tid=15054], [object_number=13, tag_tid=1, entry_tid=15053]]], state=connected] 2024-04-23T21:42:47.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:47.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=12, tag_tid=1, entry_tid=15056' 2024-04-23T21:42:47.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=12, tag_tid=1, entry_tid=15056' 2024-04-23T21:42:47.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=12, tag_tid=1, entry_tid=15056' '!=' 'object_number=12, tag_tid=1, entry_tid=15056' 2024-04-23T21:42:47.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:47.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:42:47.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_11 2024-04-23T21:42:47.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:47.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:47.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:42:47.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:47.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_11 ']' 2024-04-23T21:42:47.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_11' 2024-04-23T21:42:47.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_11 2024-04-23T21:42:47.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:47.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:47.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:47.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:47.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:47.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:47.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:47.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:47.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:47.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:47.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_11 2024-04-23T21:42:47.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:47.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:47.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:47.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:47.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:47.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:47.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:47.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:47.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:47.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:47.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:47.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:47.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:47.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:47.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:47.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:47.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:47.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:47.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:47.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:47.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:47.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_11 2024-04-23T21:42:47.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_11'] 2024-04-23T21:42:47.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:47.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:47.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:47.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_11 2024-04-23T21:42:47.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:47.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:47.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:47.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:47.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:47.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:47.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:47.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:47.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:47.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:47.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:47.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:47.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:47.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:47.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:47.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:47.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:47.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:47.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:47.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:47.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:47.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:47.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:47.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:47.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:47.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:47.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:47.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:47.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:47.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:48.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:48.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:48.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:48.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_11 2024-04-23T21:42:48.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:48.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_11 2024-04-23T21:42:48.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:48.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:48.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:42:48.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_11 '' 2024-04-23T21:42:48.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:48.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:48.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:42:48.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:48.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_11.status 2024-04-23T21:42:48.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_11.status 2024-04-23T21:42:48.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:48.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:48.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_11 2024-04-23T21:42:48.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:48.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:42:48.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:42:48.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:48.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15055], [object_number=14, tag_tid=1, entry_tid=15054]]], state=connected] 2024-04-23T21:42:48.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=15056], [object_number=15, tag_tid=1, entry_tid=15055], [object_number=14, tag_tid=1, entry_tid=15054], [object_number=13, tag_tid=1, entry_tid=15053]]], state=connected] 2024-04-23T21:42:48.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:48.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:42:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_11 2024-04-23T21:42:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:42:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_11 '..*' 2024-04-23T21:42:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:42:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_11.status 2024-04-23T21:42:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_11.status 2024-04-23T21:42:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:48.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:48.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_11 2024-04-23T21:42:48.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:48.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:42:48.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:42:48.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:48.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15055], [object_number=14, tag_tid=1, entry_tid=15054]]], state=connected] 2024-04-23T21:42:48.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=15056], [object_number=15, tag_tid=1, entry_tid=15055], [object_number=14, tag_tid=1, entry_tid=15054], [object_number=13, tag_tid=1, entry_tid=15053]]], state=connected] 2024-04-23T21:42:48.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:48.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=12, tag_tid=1, entry_tid=15056' 2024-04-23T21:42:48.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=12, tag_tid=1, entry_tid=15056' 2024-04-23T21:42:48.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=12, tag_tid=1, entry_tid=15056' '!=' 'object_number=12, tag_tid=1, entry_tid=15056' 2024-04-23T21:42:48.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:48.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:42:48.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_11 2024-04-23T21:42:48.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:48.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:48.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:42:48.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:48.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_11 ']' 2024-04-23T21:42:48.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_11' 2024-04-23T21:42:48.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_11 2024-04-23T21:42:48.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:48.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:48.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:48.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:48.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:48.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:48.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:49.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:49.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:49.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:49.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_11 2024-04-23T21:42:49.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:49.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:49.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:49.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:49.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:49.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:49.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:49.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:49.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:49.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:49.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:49.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:49.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:49.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:49.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:49.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:49.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:49.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:49.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:49.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:49.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:49.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:49.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:49.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:49.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:49.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:49.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:49.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:49.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:49.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:49.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:49.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:49.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:49.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_11 2024-04-23T21:42:49.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_11'] 2024-04-23T21:42:49.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:49.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:49.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:49.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_11 2024-04-23T21:42:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:49.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:49.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:49.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:49.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:49.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:49.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:49.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:49.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:49.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:49.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:49.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:49.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:49.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:49.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:49.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:49.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:49.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:49.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:49.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_11 2024-04-23T21:42:49.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:49.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_11 2024-04-23T21:42:49.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:49.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:49.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:42:49.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_11 '' 2024-04-23T21:42:49.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:49.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:49.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:42:49.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:49.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_11.status 2024-04-23T21:42:49.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_11.status 2024-04-23T21:42:49.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:49.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:49.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_11 2024-04-23T21:42:49.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:49.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:42:49.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:42:49.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:49.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15055], [object_number=14, tag_tid=1, entry_tid=15054]]], state=connected] 2024-04-23T21:42:49.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15055], [object_number=14, tag_tid=1, entry_tid=15054]]], state=connected] 2024-04-23T21:42:49.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:49.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:42:49.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_11 2024-04-23T21:42:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:42:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_11 '..*' 2024-04-23T21:42:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:42:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_11.status 2024-04-23T21:42:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_11.status 2024-04-23T21:42:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:49.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:49.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_11 2024-04-23T21:42:49.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:49.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:42:49.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:42:49.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:49.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15055], [object_number=14, tag_tid=1, entry_tid=15054]]], state=connected] 2024-04-23T21:42:49.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15055], [object_number=14, tag_tid=1, entry_tid=15054]]], state=connected] 2024-04-23T21:42:49.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.status 2024-04-23T21:42:49.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:42:49.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:42:49.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:49.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_11 snap 2024-04-23T21:42:49.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:49.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:49.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:42:49.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:49.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:42:49.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:42:49.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:42:50.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_11@snap 2024-04-23T21:42:50.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_11': 2024-04-23T21:42:50.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:42:50.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:42:50.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:42:50.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 120fb9104fce 2024-04-23T21:42:50.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.120fb9104fce 2024-04-23T21:42:50.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:42:50.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:42:50.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:42:50.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:42:50.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:33:38 2024 2024-04-23T21:42:50.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:33:38 2024 2024-04-23T21:42:50.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:35:52 2024 2024-04-23T21:42:50.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:42:50.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 120fb9104fce 2024-04-23T21:42:50.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:42:50.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:42:50.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: bc0440c2-44cd-4587-a1d4-ca0dafc542dd 2024-04-23T21:42:50.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:42:50.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:50.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_11 snap 2024-04-23T21:42:50.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:50.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:42:50.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:50.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:42:50.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.export 2024-04-23T21:42:50.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_11.export 2024-04-23T21:42:50.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_11.export 2024-04-23T21:42:50.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_11@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.export 2024-04-23T21:42:50.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:42:50.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_11@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_11.export 2024-04-23T21:42:50.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:42:50.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_11.export 2024-04-23T21:42:50.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_11.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_11.export 2024-04-23T21:42:50.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:50.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:42:50.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_12 2024-04-23T21:42:50.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_12 snap 2024-04-23T21:42:50.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:50.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:50.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:42:50.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:42:50.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_12@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:42:51.048 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:42:51.048+0000 7fe96bb66640 -1 bdev(0x55a5abf68e00 /var/lib/ceph/osd/cluster2-2/block) aio_submit retries 3 2024-04-23T21:42:54.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_12 2024-04-23T21:42:54.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:54.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:54.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:42:54.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_12 started 2024-04-23T21:42:54.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:54.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:54.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:42:54.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:42:54.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:42:54.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:54.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:42:55.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_12 started 2024-04-23T21:42:55.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:55.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:55.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:42:55.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:42:55.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:42:55.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:42:55.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_12 2024-04-23T21:42:55.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:42:55.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:42:55.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:55.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:55.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:42:55.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:42:55.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:55.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:42:55.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:42:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_12 2024-04-23T21:42:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:42:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:55.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:55.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:55.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:55.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:55.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:42:55.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:55.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:55.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:55.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:42:55.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:55.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:55.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:55.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:55.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:55.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:55.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:55.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:42:55.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:42:55.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:55.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:55.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:55.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:55.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_12 2024-04-23T21:42:55.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_12'] 2024-04-23T21:42:55.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:42:55.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_12 2024-04-23T21:42:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:42:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:42:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:55.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:55.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:55.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:42:55.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:55.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:55.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:55.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:55.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:42:55.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:55.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:55.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:55.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:55.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:42:55.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:42:55.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:42:55.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:55.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:42:55.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:42:55.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:42:55.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:42:55.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:55.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:55.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:42:55.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:42:55.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:42:55.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:42:55.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:55.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:55.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:55.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:55.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_12 2024-04-23T21:42:55.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:42:55.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:42:55.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:42:55.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:42:55.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:42:55.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:42:55.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:42:55.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:55.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_12 2024-04-23T21:42:55.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:42:55.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:55.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:55.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:42:55.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:42:55.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_12 2024-04-23T21:42:55.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:42:55.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:42:55.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:55.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:42:55.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:42:55.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:42:55.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:42:55.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:55.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:42:55.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_12 2024-04-23T21:42:55.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:55.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:55.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:42:55.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:55.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_12 ']' 2024-04-23T21:42:55.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_12' 2024-04-23T21:42:55.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_12 2024-04-23T21:42:55.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:55.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:55.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:55.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:55.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:55.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:55.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:55.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:55.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:55.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:55.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_12 2024-04-23T21:42:55.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:55.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:55.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:55.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:55.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:55.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:55.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:55.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:55.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:55.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:55.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_12 2024-04-23T21:42:55.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_12'] 2024-04-23T21:42:55.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:55.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:55.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:55.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_12 2024-04-23T21:42:55.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:55.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:55.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:55.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:55.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:55.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:55.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:55.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:55.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:55.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:55.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:55.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:55.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:55.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:55.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:55.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:55.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:55.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:55.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:55.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:55.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:55.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:55.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:55.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:55.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:55.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:55.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:55.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:55.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:55.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:55.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:55.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:55.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:55.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_12 2024-04-23T21:42:55.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:55.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_12 2024-04-23T21:42:55.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:55.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:55.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:42:55.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_12 '' 2024-04-23T21:42:55.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:55.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:55.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:42:55.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:55.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_12.status 2024-04-23T21:42:55.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_12.status 2024-04-23T21:42:55.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:55.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:55.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_12 2024-04-23T21:42:55.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:55.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:42:55.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:42:55.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:55.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682]]], state=connected] 2024-04-23T21:42:55.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682], [object_number=9, tag_tid=1, entry_tid=11681], [object_number=8, tag_tid=1, entry_tid=11680]]], state=connected] 2024-04-23T21:42:55.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:55.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:42:55.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_12 2024-04-23T21:42:55.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:55.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:55.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:42:55.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_12 '..*' 2024-04-23T21:42:55.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:55.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:55.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:42:55.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:55.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_12.status 2024-04-23T21:42:55.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_12.status 2024-04-23T21:42:55.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:55.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:55.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_12 2024-04-23T21:42:55.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:55.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:42:55.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:42:55.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:55.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682]]], state=connected] 2024-04-23T21:42:55.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682], [object_number=9, tag_tid=1, entry_tid=11681], [object_number=8, tag_tid=1, entry_tid=11680]]], state=connected] 2024-04-23T21:42:55.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:55.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=11, tag_tid=1, entry_tid=11683' 2024-04-23T21:42:55.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=11, tag_tid=1, entry_tid=11683' 2024-04-23T21:42:55.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=11683' '!=' '' 2024-04-23T21:42:55.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:42:55.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=11683' = '' 2024-04-23T21:42:55.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=11, tag_tid=1, entry_tid=11683' 2024-04-23T21:42:55.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:42:55.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:42:55.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:55.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:42:55.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=11, tag_tid=1, entry_tid=11683' 2024-04-23T21:42:55.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:42:55.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:55.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:42:55.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:42:55.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:42:55.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:55.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:42:55.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=11, tag_tid=1, entry_tid=11683' 2024-04-23T21:42:55.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:42:55.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:42:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=11683 2024-04-23T21:42:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 11683 2024-04-23T21:42:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:42:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:55.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:42:56.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_12 2024-04-23T21:42:56.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:56.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:56.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:42:56.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:56.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_12 ']' 2024-04-23T21:42:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_12' 2024-04-23T21:42:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_12 2024-04-23T21:42:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:56.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:56.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_12 2024-04-23T21:42:56.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:56.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:56.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:56.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:56.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:56.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:56.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:56.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:56.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:56.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:56.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:56.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:56.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:56.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:56.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:56.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:56.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:56.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:56.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:56.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:56.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:56.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:56.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:56.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:56.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:56.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:56.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:56.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:56.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:56.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:56.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:56.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:56.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:56.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_12 2024-04-23T21:42:56.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_12'] 2024-04-23T21:42:56.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:56.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_12 2024-04-23T21:42:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:56.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:56.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:56.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:56.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:56.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:56.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:56.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:56.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:56.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:56.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:56.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:56.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:56.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:56.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:56.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:56.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:56.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:56.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:56.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:56.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:56.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:56.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:56.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:56.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:56.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_12 2024-04-23T21:42:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_12 2024-04-23T21:42:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:42:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_12 '' 2024-04-23T21:42:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:42:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_12.status 2024-04-23T21:42:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_12.status 2024-04-23T21:42:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:56.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_12 2024-04-23T21:42:56.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:56.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:42:56.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:42:56.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:56.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682]]], state=connected] 2024-04-23T21:42:56.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682], [object_number=9, tag_tid=1, entry_tid=11681], [object_number=8, tag_tid=1, entry_tid=11680]]], state=connected] 2024-04-23T21:42:56.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:56.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:42:56.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_12 2024-04-23T21:42:56.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:56.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:56.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:42:56.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_12 '..*' 2024-04-23T21:42:56.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:56.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:56.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:42:56.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:56.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_12.status 2024-04-23T21:42:56.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_12.status 2024-04-23T21:42:56.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:56.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:56.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_12 2024-04-23T21:42:56.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:56.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:42:56.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:42:56.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:56.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682]]], state=connected] 2024-04-23T21:42:56.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682], [object_number=9, tag_tid=1, entry_tid=11681], [object_number=8, tag_tid=1, entry_tid=11680]]], state=connected] 2024-04-23T21:42:56.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:56.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=11, tag_tid=1, entry_tid=11683' 2024-04-23T21:42:56.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=11, tag_tid=1, entry_tid=11683' 2024-04-23T21:42:56.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=11683' '!=' 'object_number=11, tag_tid=1, entry_tid=11683' 2024-04-23T21:42:56.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:56.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:42:56.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_12 2024-04-23T21:42:56.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:56.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:56.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:42:56.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:56.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_12 ']' 2024-04-23T21:42:56.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_12' 2024-04-23T21:42:56.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_12 2024-04-23T21:42:56.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:56.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:56.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:56.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:56.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:56.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:56.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:56.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:56.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:56.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:56.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_12 2024-04-23T21:42:56.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:56.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:56.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:56.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:56.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:56.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:56.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:56.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:56.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:56.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:56.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:56.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:56.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:56.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:56.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:56.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:56.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:56.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:56.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:56.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:56.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:56.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:56.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:56.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:56.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:56.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:56.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:56.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:56.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:56.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:56.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:56.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:56.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:56.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_12 2024-04-23T21:42:56.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_12'] 2024-04-23T21:42:56.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:56.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:56.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:56.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_12 2024-04-23T21:42:56.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:56.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:56.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:56.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:56.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:56.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:56.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:56.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:56.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:56.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:56.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:56.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:56.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:56.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:56.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:56.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:56.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:56.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:56.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:57.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:57.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:57.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:57.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_12 2024-04-23T21:42:57.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:57.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_12 2024-04-23T21:42:57.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:57.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:57.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:42:57.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_12 '' 2024-04-23T21:42:57.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:57.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:42:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_12.status 2024-04-23T21:42:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_12.status 2024-04-23T21:42:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_12 2024-04-23T21:42:57.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:57.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:42:57.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:42:57.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:57.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682]]], state=connected] 2024-04-23T21:42:57.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682], [object_number=9, tag_tid=1, entry_tid=11681], [object_number=8, tag_tid=1, entry_tid=11680]]], state=connected] 2024-04-23T21:42:57.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:42:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_12 2024-04-23T21:42:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:42:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_12 '..*' 2024-04-23T21:42:57.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:57.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:57.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:42:57.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:57.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_12.status 2024-04-23T21:42:57.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_12.status 2024-04-23T21:42:57.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:57.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:57.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_12 2024-04-23T21:42:57.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:57.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:42:57.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:42:57.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:57.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682]]], state=connected] 2024-04-23T21:42:57.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682], [object_number=9, tag_tid=1, entry_tid=11681], [object_number=8, tag_tid=1, entry_tid=11680]]], state=connected] 2024-04-23T21:42:57.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:57.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=11, tag_tid=1, entry_tid=11683' 2024-04-23T21:42:57.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=11, tag_tid=1, entry_tid=11683' 2024-04-23T21:42:57.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=11683' '!=' 'object_number=11, tag_tid=1, entry_tid=11683' 2024-04-23T21:42:57.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:57.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:42:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_12 2024-04-23T21:42:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:42:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:42:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:42:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_12 ']' 2024-04-23T21:42:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_12' 2024-04-23T21:42:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_12 2024-04-23T21:42:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:42:58.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:42:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:42:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:42:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:42:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:42:58.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:42:58.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:42:58.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:58.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_12 2024-04-23T21:42:58.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:42:58.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:58.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:58.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:42:58.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:42:58.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:58.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:58.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:58.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:42:58.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:58.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:58.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:58.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:42:58.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:58.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:42:58.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:42:58.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:58.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:58.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:42:58.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:42:58.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:58.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:58.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:58.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:42:58.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:42:58.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:58.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:58.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:42:58.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:42:58.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:42:58.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:58.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:58.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:42:58.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_12 2024-04-23T21:42:58.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_12'] 2024-04-23T21:42:58.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:42:58.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:42:58.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:42:58.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_12 2024-04-23T21:42:58.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:42:58.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:42:58.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:42:58.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:42:58.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:42:58.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:42:58.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:42:58.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:42:58.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:42:58.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:58.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:42:58.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:58.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:42:58.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:58.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:42:58.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:42:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:42:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:42:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:42:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:42:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:42:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:42:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:42:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:42:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:42:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:42:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:42:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:42:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:42:58.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:42:58.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:58.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:58.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:42:58.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_12 2024-04-23T21:42:58.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:42:58.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_12 2024-04-23T21:42:58.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:58.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:58.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:42:58.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_12 '' 2024-04-23T21:42:58.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:58.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:58.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:42:58.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:42:58.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_12.status 2024-04-23T21:42:58.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_12.status 2024-04-23T21:42:58.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:58.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:58.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_12 2024-04-23T21:42:58.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:42:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:42:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682]]], state=connected] 2024-04-23T21:42:58.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682], [object_number=9, tag_tid=1, entry_tid=11681], [object_number=8, tag_tid=1, entry_tid=11680]]], state=connected] 2024-04-23T21:42:58.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:58.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:42:58.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_12 2024-04-23T21:42:58.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:58.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:58.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:42:58.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_12 '..*' 2024-04-23T21:42:58.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:42:58.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:42:58.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:42:58.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:42:58.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_12.status 2024-04-23T21:42:58.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_12.status 2024-04-23T21:42:58.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:42:58.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:58.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_12 2024-04-23T21:42:58.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:58.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:42:58.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:42:58.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:42:58.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682]]], state=connected] 2024-04-23T21:42:58.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682], [object_number=9, tag_tid=1, entry_tid=11681], [object_number=8, tag_tid=1, entry_tid=11680]]], state=connected] 2024-04-23T21:42:58.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:42:58.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=11, tag_tid=1, entry_tid=11683' 2024-04-23T21:42:58.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=11, tag_tid=1, entry_tid=11683' 2024-04-23T21:42:58.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=11683' '!=' 'object_number=11, tag_tid=1, entry_tid=11683' 2024-04-23T21:42:58.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:42:58.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:43:00.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_12 2024-04-23T21:43:00.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:00.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:00.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:43:00.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:00.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_12 ']' 2024-04-23T21:43:00.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_12' 2024-04-23T21:43:00.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_12 2024-04-23T21:43:00.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:00.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:00.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:00.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:00.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:00.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:00.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:00.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:00.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:00.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:00.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_12 2024-04-23T21:43:00.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:00.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:00.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:00.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:00.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:00.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:00.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:00.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:00.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:00.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:00.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:00.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:00.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:00.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:00.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:00.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:00.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:00.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:00.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:00.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:00.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:00.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_12 2024-04-23T21:43:00.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_12'] 2024-04-23T21:43:00.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:00.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:00.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:00.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_12 2024-04-23T21:43:00.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:43:00.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:00.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:00.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:43:00.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:43:00.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:00.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:00.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:00.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:43:00.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:00.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:00.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:00.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:43:00.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:00.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:43:00.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:43:00.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:00.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:00.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:43:00.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:43:00.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:00.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:00.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:00.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:00.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:00.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:00.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:00.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:43:00.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:00.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:43:00.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:00.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:00.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:00.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_12 2024-04-23T21:43:00.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:00.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_12 2024-04-23T21:43:00.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:00.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:00.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:43:00.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_12 '' 2024-04-23T21:43:00.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:00.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:00.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:43:00.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:00.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_12.status 2024-04-23T21:43:00.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_12.status 2024-04-23T21:43:00.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:00.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:43:00.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_12 2024-04-23T21:43:00.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:43:00.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:00.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:00.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:00.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682]]], state=connected] 2024-04-23T21:43:00.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682]]], state=connected] 2024-04-23T21:43:00.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:43:00.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_12 2024-04-23T21:43:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:43:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_12 '..*' 2024-04-23T21:43:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:43:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_12.status 2024-04-23T21:43:00.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_12.status 2024-04-23T21:43:00.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:00.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:43:00.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_12 2024-04-23T21:43:00.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:43:00.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:00.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:00.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:00.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682]]], state=connected] 2024-04-23T21:43:00.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11683], [object_number=10, tag_tid=1, entry_tid=11682]]], state=connected] 2024-04-23T21:43:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.status 2024-04-23T21:43:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:00.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_12 snap 2024-04-23T21:43:00.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:00.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:00.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:43:00.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:00.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:43:00.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:43:00.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:43:01.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_12@snap 2024-04-23T21:43:01.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_12': 2024-04-23T21:43:01.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:43:01.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:43:01.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:43:01.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 120f492194a 2024-04-23T21:43:01.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.120f492194a 2024-04-23T21:43:01.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:43:01.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:43:01.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:43:01.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:43:01.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:33:51 2024 2024-04-23T21:43:01.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:33:51 2024 2024-04-23T21:43:01.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:35:53 2024 2024-04-23T21:43:01.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:43:01.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 120f492194a 2024-04-23T21:43:01.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:43:01.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:43:01.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 62d75ba2-c8ef-4af4-a5b3-ad3d8f8370ab 2024-04-23T21:43:01.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:43:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_12 snap 2024-04-23T21:43:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:43:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:43:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.export 2024-04-23T21:43:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_12.export 2024-04-23T21:43:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_12.export 2024-04-23T21:43:01.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_12@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.export 2024-04-23T21:43:01.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:43:01.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_12@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_12.export 2024-04-23T21:43:02.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:43:02.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_12.export 2024-04-23T21:43:02.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_12.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_12.export 2024-04-23T21:43:02.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:02.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:43:02.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_13 2024-04-23T21:43:02.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_13 snap 2024-04-23T21:43:02.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:43:02.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:02.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:43:02.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:02.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_13@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:43:04.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_13 2024-04-23T21:43:04.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:04.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:43:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_13 started 2024-04-23T21:43:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:43:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:43:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:43:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:04.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:43:05.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_13 started 2024-04-23T21:43:05.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:05.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:05.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:43:05.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:43:05.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:43:05.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:43:05.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_13 2024-04-23T21:43:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:43:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:43:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:43:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:43:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:43:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:43:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:43:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_13 2024-04-23T21:43:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:43:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:43:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:43:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:43:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:43:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:05.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:43:05.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:43:05.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:43:05.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:05.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:43:05.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:43:05.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:43:05.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:43:05.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:05.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:05.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:43:05.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:43:05.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:43:05.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:43:05.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:05.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:05.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:05.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:05.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_13 2024-04-23T21:43:05.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:43:05.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:43:05.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:43:05.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:43:05.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:43:05.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:43:05.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:43:05.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:05.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_13 2024-04-23T21:43:05.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:43:05.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:43:05.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:05.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:43:05.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:43:05.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_13 2024-04-23T21:43:05.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:43:05.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:43:05.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:05.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:43:05.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:43:05.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:43:05.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:43:05.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:05.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:43:05.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_13 2024-04-23T21:43:05.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:05.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:05.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:43:05.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:05.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_13 ']' 2024-04-23T21:43:05.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_13' 2024-04-23T21:43:05.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_13 2024-04-23T21:43:05.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:05.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:05.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:05.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:05.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:05.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:05.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:05.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:05.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:05.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:05.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_13 2024-04-23T21:43:05.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:05.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:05.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:05.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:05.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:05.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:05.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:05.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:05.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:05.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:05.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:05.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:05.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:05.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:05.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:05.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:05.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:05.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:05.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:05.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:05.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:05.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:05.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:05.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:05.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:05.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:05.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:05.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:05.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:05.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:05.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:05.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:05.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:05.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_13 2024-04-23T21:43:05.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:05.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_13 2024-04-23T21:43:05.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:05.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:05.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:05.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_13 '' 2024-04-23T21:43:05.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:05.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:05.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:05.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:05.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_13.status 2024-04-23T21:43:05.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_13.status 2024-04-23T21:43:05.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:05.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:05.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_13 2024-04-23T21:43:05.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:05.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:05.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:05.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:05.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11890], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:05.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11890], [object_number=9, tag_tid=1, entry_tid=11889], [object_number=8, tag_tid=1, entry_tid=11888], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:05.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:05.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_13 2024-04-23T21:43:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_13 '..*' 2024-04-23T21:43:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_13.status 2024-04-23T21:43:05.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_13.status 2024-04-23T21:43:05.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:05.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:05.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_13 2024-04-23T21:43:05.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:05.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:05.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:05.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:05.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11890], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:05.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11890], [object_number=9, tag_tid=1, entry_tid=11889], [object_number=8, tag_tid=1, entry_tid=11888], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:05.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:05.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:05.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:05.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=11890' '!=' '' 2024-04-23T21:43:05.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:43:05.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=11890' = '' 2024-04-23T21:43:05.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:05.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:05.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:43:05.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:43:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:43:05.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:05.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:43:05.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:05.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:43:05.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:05.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:43:05.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:05.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:43:05.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:05.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=11890 2024-04-23T21:43:05.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 11890 2024-04-23T21:43:05.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:43:05.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:05.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:43:05.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_13 2024-04-23T21:43:05.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:05.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:05.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:43:05.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:05.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_13 ']' 2024-04-23T21:43:05.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_13' 2024-04-23T21:43:05.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_13 2024-04-23T21:43:05.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:05.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:05.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:05.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:05.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:05.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:05.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:05.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:05.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:05.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:05.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_13 2024-04-23T21:43:05.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:05.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:05.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:05.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:05.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:05.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:05.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:05.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:05.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:05.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:05.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:05.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:05.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:05.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:05.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:05.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:05.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:05.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:05.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:05.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:05.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:05.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:05.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:05.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:05.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:05.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:05.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:05.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:05.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:05.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:05.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:05.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:05.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:05.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_13 2024-04-23T21:43:05.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_13 2024-04-23T21:43:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_13 '' 2024-04-23T21:43:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_13.status 2024-04-23T21:43:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_13.status 2024-04-23T21:43:05.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_13 2024-04-23T21:43:05.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:05.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:05.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:05.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:05.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11890], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:05.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11890], [object_number=9, tag_tid=1, entry_tid=11889], [object_number=8, tag_tid=1, entry_tid=11888], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:05.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:05.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_13 2024-04-23T21:43:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_13 '..*' 2024-04-23T21:43:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_13.status 2024-04-23T21:43:05.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_13.status 2024-04-23T21:43:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_13 2024-04-23T21:43:05.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:05.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:05.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:05.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:05.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11890], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:05.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11890], [object_number=9, tag_tid=1, entry_tid=11889], [object_number=8, tag_tid=1, entry_tid=11888], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:05.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:05.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:05.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:05.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=11890' '!=' 'object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:05.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:06.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:43:06.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_13 2024-04-23T21:43:06.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:06.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:06.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:43:06.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:06.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_13 ']' 2024-04-23T21:43:06.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_13' 2024-04-23T21:43:06.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_13 2024-04-23T21:43:06.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:06.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:06.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:06.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:06.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:06.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:06.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:06.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:06.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:06.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:06.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_13 2024-04-23T21:43:06.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:06.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:06.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:06.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:06.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:06.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:06.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:06.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:06.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:06.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:06.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:06.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:06.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:06.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:06.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:06.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:06.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:06.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:06.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:06.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:06.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:06.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:06.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:06.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:06.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:06.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:06.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:06.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:06.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:06.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:06.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:06.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:06.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:06.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_13 2024-04-23T21:43:06.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:06.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_13 2024-04-23T21:43:06.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:06.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_13 '' 2024-04-23T21:43:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_13.status 2024-04-23T21:43:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_13.status 2024-04-23T21:43:06.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_13 2024-04-23T21:43:06.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:06.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:06.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:06.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:06.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11890], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:06.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11890], [object_number=9, tag_tid=1, entry_tid=11889], [object_number=8, tag_tid=1, entry_tid=11888], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:06.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:06.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:06.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_13 2024-04-23T21:43:06.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:06.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:06.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:06.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_13 '..*' 2024-04-23T21:43:06.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:06.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:06.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:06.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:06.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_13.status 2024-04-23T21:43:06.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_13.status 2024-04-23T21:43:06.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:06.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:06.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_13 2024-04-23T21:43:06.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:06.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:06.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:06.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:06.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11890], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:06.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11890], [object_number=9, tag_tid=1, entry_tid=11889], [object_number=8, tag_tid=1, entry_tid=11888], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:06.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:06.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:06.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:06.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=11890' '!=' 'object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:06.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:06.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:43:07.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_13 2024-04-23T21:43:07.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:07.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:07.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:43:07.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:07.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_13 ']' 2024-04-23T21:43:07.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_13' 2024-04-23T21:43:07.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_13 2024-04-23T21:43:07.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:07.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:07.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:07.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:07.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:07.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:07.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:07.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:07.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:07.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:07.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_13 2024-04-23T21:43:07.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:07.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:07.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:07.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:07.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:07.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:07.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:07.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:07.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:07.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:07.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:07.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:07.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:07.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:07.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:07.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:07.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:07.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:07.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:07.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:07.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:07.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:07.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:07.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:07.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:07.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:07.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:07.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:07.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:07.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_13 2024-04-23T21:43:07.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:07.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_13 2024-04-23T21:43:07.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:07.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:07.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:07.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_13 '' 2024-04-23T21:43:07.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:07.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:07.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:07.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:07.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_13.status 2024-04-23T21:43:07.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_13.status 2024-04-23T21:43:07.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:07.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:07.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_13 2024-04-23T21:43:07.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:07.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:07.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:07.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:07.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11890], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:07.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11890], [object_number=9, tag_tid=1, entry_tid=11889], [object_number=8, tag_tid=1, entry_tid=11888], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:07.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:07.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:07.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_13 2024-04-23T21:43:07.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:07.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:07.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:07.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_13 '..*' 2024-04-23T21:43:07.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:07.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:07.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:07.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:07.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_13.status 2024-04-23T21:43:07.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_13.status 2024-04-23T21:43:07.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:07.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:07.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_13 2024-04-23T21:43:07.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:07.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:07.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:07.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:07.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11890], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:07.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11890], [object_number=9, tag_tid=1, entry_tid=11889], [object_number=8, tag_tid=1, entry_tid=11888], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:07.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:07.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:07.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:07.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=11890' '!=' 'object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:07.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:07.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:43:09.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_13 2024-04-23T21:43:09.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:09.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:09.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:43:09.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:09.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_13 ']' 2024-04-23T21:43:09.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_13' 2024-04-23T21:43:09.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_13 2024-04-23T21:43:09.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:09.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:09.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:09.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:09.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:09.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:09.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:09.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:09.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:09.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:09.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_13 2024-04-23T21:43:09.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:09.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:09.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:09.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:09.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:09.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:09.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:09.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:09.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:09.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:09.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:09.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:09.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:09.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:09.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:09.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:09.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:09.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:09.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:09.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:09.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:09.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:09.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:09.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:09.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:09.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:09.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:09.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:09.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:09.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:09.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:09.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:09.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:09.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_13 2024-04-23T21:43:09.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:09.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_13 2024-04-23T21:43:09.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:09.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:09.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:09.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_13 '' 2024-04-23T21:43:09.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:09.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:09.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:09.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:09.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_13.status 2024-04-23T21:43:09.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_13.status 2024-04-23T21:43:09.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:09.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:09.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_13 2024-04-23T21:43:09.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:09.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:09.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:09.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:09.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11890], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:09.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11890], [object_number=9, tag_tid=1, entry_tid=11889], [object_number=8, tag_tid=1, entry_tid=11888], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:09.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:09.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:09.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_13 2024-04-23T21:43:09.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:09.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:09.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:09.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_13 '..*' 2024-04-23T21:43:09.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:09.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:09.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:09.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:09.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_13.status 2024-04-23T21:43:09.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_13.status 2024-04-23T21:43:09.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:09.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:09.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_13 2024-04-23T21:43:09.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:09.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:09.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:09.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:09.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11890], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:09.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11890], [object_number=9, tag_tid=1, entry_tid=11889], [object_number=8, tag_tid=1, entry_tid=11888], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:09.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:09.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:09.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:09.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=11890' '!=' 'object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:09.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:09.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:43:11.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_13 2024-04-23T21:43:11.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:11.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:11.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:43:11.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:11.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_13 ']' 2024-04-23T21:43:11.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_13' 2024-04-23T21:43:11.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_13 2024-04-23T21:43:11.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:11.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:11.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:11.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:11.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:11.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:11.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:11.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:11.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:11.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:11.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_13 2024-04-23T21:43:11.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:11.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:11.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:11.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:11.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:11.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:11.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:11.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:11.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:11.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:11.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:11.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:11.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:11.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:11.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:11.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:11.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:11.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:11.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:11.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:11.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:11.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:11.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:11.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:11.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:11.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:11.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:11.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:11.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:11.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:11.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:11.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:11.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:11.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_13 2024-04-23T21:43:11.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_13 2024-04-23T21:43:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_13 '' 2024-04-23T21:43:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_13.status 2024-04-23T21:43:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_13.status 2024-04-23T21:43:11.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:11.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:11.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_13 2024-04-23T21:43:11.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:11.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:11.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:11.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:11.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11890], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:11.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11890], [object_number=9, tag_tid=1, entry_tid=11889], [object_number=8, tag_tid=1, entry_tid=11888], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:11.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:11.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:11.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_13 2024-04-23T21:43:11.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:11.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:11.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:11.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_13 '..*' 2024-04-23T21:43:11.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:11.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:11.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:11.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:11.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_13.status 2024-04-23T21:43:11.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_13.status 2024-04-23T21:43:11.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:11.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:11.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_13 2024-04-23T21:43:11.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:11.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:11.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:11.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:11.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11890], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:11.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11890], [object_number=9, tag_tid=1, entry_tid=11889], [object_number=8, tag_tid=1, entry_tid=11888], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:11.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:11.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:11.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:11.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=11890' '!=' 'object_number=10, tag_tid=1, entry_tid=11890' 2024-04-23T21:43:11.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:11.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:43:13.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_13 2024-04-23T21:43:13.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:13.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:13.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:43:13.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:13.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_13 ']' 2024-04-23T21:43:13.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_13' 2024-04-23T21:43:13.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_13 2024-04-23T21:43:13.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:13.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:13.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:13.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:13.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:13.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:13.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:13.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:13.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:13.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:13.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_13 2024-04-23T21:43:13.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:13.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:13.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:13.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:13.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:13.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:13.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:13.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:13.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:13.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:13.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:13.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:13.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:13.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:13.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:13.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:13.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:13.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:13.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:13.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:13.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:13.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:13.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:13.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:13.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:13.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:13.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:13.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:13.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:13.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:13.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:13.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:13.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:13.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_13 2024-04-23T21:43:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_13 2024-04-23T21:43:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_13 '' 2024-04-23T21:43:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_13.status 2024-04-23T21:43:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_13.status 2024-04-23T21:43:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:13.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:13.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_13 2024-04-23T21:43:13.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:13.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:13.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:13.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:13.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11890], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:13.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11890], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:13.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:13.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:13.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_13 2024-04-23T21:43:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_13 '..*' 2024-04-23T21:43:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:43:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_13.status 2024-04-23T21:43:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_13.status 2024-04-23T21:43:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_13 2024-04-23T21:43:13.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:13.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:13.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:13.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:13.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11890], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:13.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11890], [object_number=11, tag_tid=1, entry_tid=11887]]], state=connected] 2024-04-23T21:43:13.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.status 2024-04-23T21:43:13.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_13 snap 2024-04-23T21:43:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:43:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:43:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:43:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:43:14.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_13@snap 2024-04-23T21:43:14.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_13': 2024-04-23T21:43:14.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:43:14.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:43:14.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:43:14.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 11b883978e04 2024-04-23T21:43:14.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.11b883978e04 2024-04-23T21:43:14.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:43:14.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:43:14.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:43:14.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:43:14.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:34:01 2024 2024-04-23T21:43:14.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:34:01 2024 2024-04-23T21:43:14.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:35:02 2024 2024-04-23T21:43:14.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:43:14.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 11b883978e04 2024-04-23T21:43:14.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:43:14.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:43:14.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: d0d62491-da49-4e1f-ac74-2a9b30a73b64 2024-04-23T21:43:14.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:43:14.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:14.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_13 snap 2024-04-23T21:43:14.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:14.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:43:14.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:14.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:43:14.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.export 2024-04-23T21:43:14.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_13.export 2024-04-23T21:43:14.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_13.export 2024-04-23T21:43:14.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_13@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.export 2024-04-23T21:43:15.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:43:15.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_13@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_13.export 2024-04-23T21:43:15.266 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:43:15.264+0000 7fc3f688f640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 2 2024-04-23T21:43:15.266 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:43:15.264+0000 7fc3f7090640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 3 2024-04-23T21:43:15.266 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:43:15.265+0000 7fc3fd09c640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 5 2024-04-23T21:43:15.267 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:43:15.265+0000 7fc3fb899640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 7 2024-04-23T21:43:15.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:43:15.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_13.export 2024-04-23T21:43:15.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_13.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_13.export 2024-04-23T21:43:15.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:15.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:43:15.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_14 2024-04-23T21:43:15.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_14 snap 2024-04-23T21:43:15.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:43:15.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:15.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:43:15.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:15.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_14@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:43:18.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_14 2024-04-23T21:43:18.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:18.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:18.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:43:18.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_14 started 2024-04-23T21:43:18.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:18.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:18.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:43:18.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:43:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:43:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:43:19.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_14 started 2024-04-23T21:43:19.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:19.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:19.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:43:19.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:43:19.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:43:19.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:43:19.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_14 2024-04-23T21:43:19.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:43:19.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:43:19.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:43:19.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:19.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:43:19.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:43:19.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:19.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:43:19.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:19.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:43:19.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:19.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_14 2024-04-23T21:43:19.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:43:19.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:43:19.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:19.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:19.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:19.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:43:19.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:19.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:19.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:19.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:19.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:43:19.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:43:19.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:19.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:19.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:19.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:43:19.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:43:19.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:43:19.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:19.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:43:19.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:43:19.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:43:19.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:43:19.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:19.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:19.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:43:19.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:43:19.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:43:19.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:43:19.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:19.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:19.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:19.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:19.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_14 2024-04-23T21:43:19.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2024-04-23T21:43:19.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:43:19.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_14 2024-04-23T21:43:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:43:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:43:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:43:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:43:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:43:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:43:19.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:43:19.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:43:19.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:19.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:43:19.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:43:19.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:43:19.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:43:19.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:43:19.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:43:19.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:43:19.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:43:19.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:43:19.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:19.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:19.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:43:19.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:43:19.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:43:19.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:43:19.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:43:19.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:19.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:19.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:19.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_14 2024-04-23T21:43:19.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2024-04-23T21:43:19.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:43:19.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:19.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:19.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_14 2024-04-23T21:43:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:43:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:43:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:43:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:43:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:43:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:43:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:43:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:43:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:43:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:43:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:43:19.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:43:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:43:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:43:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:43:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:43:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:43:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:43:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:43:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:43:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:43:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:43:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:43:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:43:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:43:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:43:19.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:43:19.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:19.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:19.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:19.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_14 2024-04-23T21:43:19.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:43:19.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:43:19.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:43:19.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:43:19.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:43:19.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:43:19.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:43:19.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:19.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_14 2024-04-23T21:43:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:43:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:43:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:43:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:43:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_14 2024-04-23T21:43:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:43:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:43:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:43:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:43:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:43:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:43:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:19.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:43:19.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_14 2024-04-23T21:43:19.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:19.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:19.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:43:19.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:19.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_14 ']' 2024-04-23T21:43:19.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_14' 2024-04-23T21:43:19.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_14 2024-04-23T21:43:19.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:19.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:19.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:19.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:19.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:19.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:19.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:19.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:19.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:19.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:19.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_14 2024-04-23T21:43:19.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:19.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:19.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:19.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:19.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:19.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:19.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:19.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:19.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:19.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:19.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:19.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:19.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:19.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:19.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:19.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:19.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:19.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:19.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:19.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:19.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:19.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:19.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:19.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:19.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:19.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:19.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:19.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:19.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:19.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:19.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:19.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:19.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:19.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:19.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2024-04-23T21:43:19.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:19.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:19.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:19.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_14 2024-04-23T21:43:19.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:43:19.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:19.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:19.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:43:19.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:43:19.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:19.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:19.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:19.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:43:19.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:19.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:19.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:19.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:43:19.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:19.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:43:19.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:43:19.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:19.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:19.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:43:19.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:43:19.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:19.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:19.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:19.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:19.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:19.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:19.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:19.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:43:19.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:19.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:43:19.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:19.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:19.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:19.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:19.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2024-04-23T21:43:19.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:19.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:19.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:19.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_14 2024-04-23T21:43:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:43:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:43:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:43:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:43:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:43:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:43:19.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:43:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:43:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:43:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:43:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:43:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:43:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:43:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:43:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:43:19.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:43:19.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:19.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:19.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:19.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:19.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:19.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_14 2024-04-23T21:43:19.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:19.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:19.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:19.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_14 '' 2024-04-23T21:43:19.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:19.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:19.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:19.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:19.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_14.status 2024-04-23T21:43:19.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_14.status 2024-04-23T21:43:19.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:19.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:19.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_14 2024-04-23T21:43:19.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:19.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:19.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:19.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:19.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:19.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11209], [object_number=8, tag_tid=1, entry_tid=11208], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:19.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:19.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:19.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_14 2024-04-23T21:43:19.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:19.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:19.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:19.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_14 '..*' 2024-04-23T21:43:19.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:19.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:19.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:19.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:19.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_14.status 2024-04-23T21:43:19.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_14.status 2024-04-23T21:43:19.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:19.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:19.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_14 2024-04-23T21:43:19.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:20.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:20.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:20.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:20.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:20.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11209], [object_number=8, tag_tid=1, entry_tid=11208], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:20.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:20.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:20.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:20.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=11209' '!=' '' 2024-04-23T21:43:20.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:43:20.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=11209' = '' 2024-04-23T21:43:20.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:20.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:20.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:43:20.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:20.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:43:20.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:20.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:43:20.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:20.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:43:20.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:20.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:43:20.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:20.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:43:20.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:20.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:43:20.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:20.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=11209 2024-04-23T21:43:20.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 11209 2024-04-23T21:43:20.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:43:20.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:20.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:43:20.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_14 2024-04-23T21:43:20.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:43:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_14 ']' 2024-04-23T21:43:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_14' 2024-04-23T21:43:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_14 2024-04-23T21:43:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:20.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:20.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_14 2024-04-23T21:43:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:20.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:20.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:20.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:20.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:20.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:20.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:20.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2024-04-23T21:43:20.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:20.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:20.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:20.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_14 2024-04-23T21:43:20.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:43:20.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:43:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:43:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:43:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:43:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:20.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:43:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:43:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:43:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:43:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:43:20.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:20.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:43:20.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:20.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:20.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:20.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:20.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2024-04-23T21:43:20.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:20.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:20.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:20.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_14 2024-04-23T21:43:20.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:43:20.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:20.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:20.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:43:20.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:43:20.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:20.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:20.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:20.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:43:20.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:43:20.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:20.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:20.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:43:20.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:43:20.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:43:20.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:43:20.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:20.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:20.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:43:20.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:43:20.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:20.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:20.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:20.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:43:20.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:43:20.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:20.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:20.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:43:20.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:43:20.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:43:20.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:20.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:20.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:20.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:20.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:20.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_14 2024-04-23T21:43:20.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:20.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:20.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:20.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_14 '' 2024-04-23T21:43:20.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:20.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:20.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:20.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:20.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_14.status 2024-04-23T21:43:20.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_14.status 2024-04-23T21:43:20.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:20.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:20.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_14 2024-04-23T21:43:20.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:20.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:20.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:20.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:20.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:20.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11209], [object_number=8, tag_tid=1, entry_tid=11208], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:20.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:20.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:20.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_14 2024-04-23T21:43:20.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:20.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:20.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:20.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_14 '..*' 2024-04-23T21:43:20.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:20.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:20.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:20.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:20.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_14.status 2024-04-23T21:43:20.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_14.status 2024-04-23T21:43:20.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:20.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:20.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_14 2024-04-23T21:43:20.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:20.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:20.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:20.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:20.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:20.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11209], [object_number=8, tag_tid=1, entry_tid=11208], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:20.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:20.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:20.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:20.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=11209' '!=' 'object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:20.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:20.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:43:21.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_14 2024-04-23T21:43:21.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:21.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:21.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:43:21.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:21.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_14 ']' 2024-04-23T21:43:21.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_14' 2024-04-23T21:43:21.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_14 2024-04-23T21:43:21.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:21.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:21.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:21.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:21.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:21.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:21.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:21.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:21.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:21.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:21.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_14 2024-04-23T21:43:21.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:21.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:21.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:21.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:21.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:21.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:21.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:21.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:21.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:21.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:21.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:21.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:21.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:21.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:21.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:21.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:21.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:21.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:21.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:21.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:21.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2024-04-23T21:43:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:21.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_14 2024-04-23T21:43:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:43:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:43:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:43:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:43:21.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:43:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:43:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:43:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:43:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:43:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:21.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:21.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:21.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:21.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:21.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:43:21.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:21.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:43:21.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:21.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:21.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:21.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:21.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2024-04-23T21:43:21.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:21.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:21.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:21.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_14 2024-04-23T21:43:21.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:43:21.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:21.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:21.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:43:21.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:43:21.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:21.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:21.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:21.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:43:21.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:43:21.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:21.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:21.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:43:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:43:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:43:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:43:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:43:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:43:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:43:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:43:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:43:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:43:21.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:43:21.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:21.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:21.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:21.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_14 2024-04-23T21:43:21.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:21.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:21.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:21.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_14 '' 2024-04-23T21:43:21.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:21.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:21.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:21.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:21.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_14.status 2024-04-23T21:43:21.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_14.status 2024-04-23T21:43:21.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:21.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:21.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_14 2024-04-23T21:43:21.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:21.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:21.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:21.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:21.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:21.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11209], [object_number=8, tag_tid=1, entry_tid=11208], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:21.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:21.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:21.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_14 2024-04-23T21:43:21.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:21.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:21.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:21.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_14 '..*' 2024-04-23T21:43:21.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:21.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:21.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:21.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:21.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_14.status 2024-04-23T21:43:21.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_14.status 2024-04-23T21:43:21.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:21.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:21.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_14 2024-04-23T21:43:21.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:21.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:21.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:21.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:21.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:21.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11209], [object_number=8, tag_tid=1, entry_tid=11208], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:21.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:21.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:21.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:21.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=11209' '!=' 'object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:21.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:21.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:43:22.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_14 2024-04-23T21:43:22.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:22.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:22.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:43:22.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:22.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_14 ']' 2024-04-23T21:43:22.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_14' 2024-04-23T21:43:22.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_14 2024-04-23T21:43:22.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:22.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:22.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:22.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:22.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:22.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:22.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:22.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_14 2024-04-23T21:43:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:22.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:22.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:22.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:22.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:22.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2024-04-23T21:43:22.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:22.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:22.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:22.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_14 2024-04-23T21:43:22.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:43:22.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:22.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:22.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:43:22.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:43:22.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:22.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:22.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:43:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:43:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:43:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:43:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:43:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:43:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:43:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:43:22.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:22.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:22.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:22.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:22.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2024-04-23T21:43:22.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:22.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:22.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_14 2024-04-23T21:43:22.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:43:22.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:22.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:22.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:43:22.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:43:22.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:22.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:22.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:22.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:43:22.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:43:22.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:22.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:22.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:43:22.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:43:22.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:43:22.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:43:22.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:22.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:43:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:43:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:43:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:43:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:43:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:43:22.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:43:22.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_14 2024-04-23T21:43:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_14 '' 2024-04-23T21:43:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_14.status 2024-04-23T21:43:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_14.status 2024-04-23T21:43:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:22.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_14 2024-04-23T21:43:22.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11209], [object_number=8, tag_tid=1, entry_tid=11208], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:22.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:22.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_14 2024-04-23T21:43:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_14 '..*' 2024-04-23T21:43:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_14.status 2024-04-23T21:43:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_14.status 2024-04-23T21:43:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:22.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:22.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_14 2024-04-23T21:43:22.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:22.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:22.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:22.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:22.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:22.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11209], [object_number=8, tag_tid=1, entry_tid=11208], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:22.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:22.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:22.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:22.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=11209' '!=' 'object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:22.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:22.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:43:24.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_14 2024-04-23T21:43:24.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:24.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:24.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:43:24.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:24.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_14 ']' 2024-04-23T21:43:24.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_14' 2024-04-23T21:43:24.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_14 2024-04-23T21:43:24.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:24.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:24.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_14 2024-04-23T21:43:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:24.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:24.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:24.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:24.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:24.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:24.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:24.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:24.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2024-04-23T21:43:24.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:24.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:24.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:24.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_14 2024-04-23T21:43:24.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:43:24.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:24.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:24.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:43:24.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:43:24.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:24.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:24.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:24.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:43:24.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:24.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:24.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:24.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:43:24.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:24.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:43:24.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:43:24.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:24.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:24.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:43:24.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:43:24.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:24.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:24.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:24.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:24.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:24.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:24.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:24.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:43:24.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:24.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:43:24.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:24.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:24.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:24.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:24.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2024-04-23T21:43:24.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:24.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:24.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:24.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_14 2024-04-23T21:43:24.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:43:24.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:24.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:24.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:43:24.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:43:24.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:24.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:24.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:24.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:43:24.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:43:24.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:24.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:24.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:43:24.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:43:24.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:43:24.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:43:24.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:24.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:24.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:43:24.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:43:24.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:24.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:24.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:24.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:43:24.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:43:24.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:24.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:24.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:43:24.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:43:24.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:43:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:24.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:24.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:24.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_14 2024-04-23T21:43:24.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:24.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:24.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:24.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_14 '' 2024-04-23T21:43:24.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:24.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:24.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:24.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:24.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_14.status 2024-04-23T21:43:24.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_14.status 2024-04-23T21:43:24.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:24.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:24.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_14 2024-04-23T21:43:24.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:24.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:24.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:24.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:24.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:24.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11209], [object_number=8, tag_tid=1, entry_tid=11208], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:24.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_14 2024-04-23T21:43:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:24.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_14 '..*' 2024-04-23T21:43:24.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:24.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:24.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:24.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:24.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_14.status 2024-04-23T21:43:24.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_14.status 2024-04-23T21:43:24.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:24.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:24.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_14 2024-04-23T21:43:24.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:24.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:24.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:24.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:24.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:24.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=11209], [object_number=8, tag_tid=1, entry_tid=11208], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:24.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:24.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:24.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:24.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=11209' '!=' 'object_number=9, tag_tid=1, entry_tid=11209' 2024-04-23T21:43:24.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:24.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:43:26.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_14 2024-04-23T21:43:26.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:26.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:26.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:43:26.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:26.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_14 ']' 2024-04-23T21:43:26.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_14' 2024-04-23T21:43:26.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_14 2024-04-23T21:43:26.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:26.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:26.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:26.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:26.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:26.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:26.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:26.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:26.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:26.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:26.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_14 2024-04-23T21:43:26.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:26.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:26.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:26.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:26.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:26.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:26.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:26.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:26.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:26.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:26.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:26.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:26.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:26.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:26.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:26.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:26.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:26.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:26.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:26.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:26.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2024-04-23T21:43:26.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:26.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_14 2024-04-23T21:43:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:43:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:43:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:43:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:26.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:43:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:43:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:43:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:43:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:43:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:43:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:26.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:26.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:26.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:26.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:26.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:26.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:26.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:43:26.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:26.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:43:26.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:26.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:26.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:26.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:26.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2024-04-23T21:43:26.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:26.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:26.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:26.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_14 2024-04-23T21:43:26.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:43:26.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:26.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:26.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:43:26.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:43:26.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:26.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:26.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:26.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:43:26.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:43:26.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:26.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:26.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:43:26.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:43:26.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:43:26.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:43:26.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:26.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:26.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:43:26.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:43:26.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:26.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:26.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:26.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:43:26.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:43:26.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:26.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:26.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:43:26.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:43:26.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:43:26.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:26.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:26.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:43:26.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_14 2024-04-23T21:43:27.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:27.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_14 2024-04-23T21:43:27.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:27.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:27.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:27.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_14 '' 2024-04-23T21:43:27.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:27.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:27.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:27.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:27.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_14.status 2024-04-23T21:43:27.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_14.status 2024-04-23T21:43:27.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:27.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:27.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_14 2024-04-23T21:43:27.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:27.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:27.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:27.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:27.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:27.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:27.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:27.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:27.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_14 2024-04-23T21:43:27.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:27.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:27.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:27.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_14 '..*' 2024-04-23T21:43:27.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:27.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:27.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:43:27.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:27.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_14.status 2024-04-23T21:43:27.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_14.status 2024-04-23T21:43:27.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:27.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:27.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_14 2024-04-23T21:43:27.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:27.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:43:27.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:43:27.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:27.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:27.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11207], [object_number=10, tag_tid=1, entry_tid=11206]]], state=connected] 2024-04-23T21:43:27.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.status 2024-04-23T21:43:27.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:27.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:43:27.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:27.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_14 snap 2024-04-23T21:43:27.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:27.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:27.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:43:27.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:27.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:43:27.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:43:27.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:43:28.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_14@snap 2024-04-23T21:43:28.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_14': 2024-04-23T21:43:28.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:43:28.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:43:28.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:43:28.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 116ed98ca136 2024-04-23T21:43:28.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.116ed98ca136 2024-04-23T21:43:28.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:43:28.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:43:28.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:43:28.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:43:28.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:34:13 2024 2024-04-23T21:43:28.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:34:13 2024 2024-04-23T21:43:28.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:35:14 2024 2024-04-23T21:43:28.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:43:28.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 116ed98ca136 2024-04-23T21:43:28.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:43:28.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:43:28.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: da53d3c5-7d2f-44c2-99f8-4c19141f4790 2024-04-23T21:43:28.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:43:28.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:28.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_14 snap 2024-04-23T21:43:28.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:28.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:43:28.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:28.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:43:28.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.export 2024-04-23T21:43:28.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_14.export 2024-04-23T21:43:28.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_14.export 2024-04-23T21:43:28.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_14@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.export 2024-04-23T21:43:28.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:43:28.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_14@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_14.export 2024-04-23T21:43:28.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:43:28.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_14.export 2024-04-23T21:43:28.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_14.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_14.export 2024-04-23T21:43:28.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:28.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:43:28.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_15 2024-04-23T21:43:28.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_15 snap 2024-04-23T21:43:28.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:43:28.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:28.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:43:28.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:28.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_15@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:43:29.104 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:43:29.104+0000 7f83eb358640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 3 2024-04-23T21:43:30.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_15 2024-04-23T21:43:30.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:30.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:30.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:43:30.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_15 started 2024-04-23T21:43:30.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:30.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:30.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:43:30.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:43:30.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:43:30.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:30.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:43:31.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_15 started 2024-04-23T21:43:31.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:43:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:43:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:43:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:43:31.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_15 2024-04-23T21:43:31.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:43:31.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:43:31.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:43:31.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:31.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:43:31.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:43:31.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:31.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:43:31.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:31.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:43:31.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:31.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_15 2024-04-23T21:43:31.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:43:31.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:43:31.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:31.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:31.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:31.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:43:31.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:31.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:31.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:31.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:31.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:43:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:43:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:43:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:43:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:43:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:43:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:43:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:43:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:43:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:31.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:43:31.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:43:31.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:43:31.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:43:31.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:31.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:31.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:31.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:31.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_15 2024-04-23T21:43:31.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:43:31.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:43:31.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:43:31.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:43:31.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:43:31.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:43:31.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:43:31.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:31.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_15 2024-04-23T21:43:31.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:43:31.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:43:31.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:31.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:43:31.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:43:31.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_15 2024-04-23T21:43:31.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:43:31.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:43:31.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:31.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:43:31.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:43:31.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:43:31.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:43:31.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:31.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:43:31.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_15 2024-04-23T21:43:31.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:43:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_15 ']' 2024-04-23T21:43:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_15' 2024-04-23T21:43:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_15 2024-04-23T21:43:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:31.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:31.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:31.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:31.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_15 2024-04-23T21:43:31.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:31.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:31.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:31.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:31.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:31.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:31.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:31.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:31.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:31.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:31.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:31.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:31.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:31.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:31.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:31.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:31.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:31.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:31.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:31.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:31.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:31.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:31.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:31.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:31.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:31.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:31.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:31.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:31.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:31.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:31.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:31.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:31.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:31.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_15 2024-04-23T21:43:31.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:31.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_15 2024-04-23T21:43:31.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:31.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:31.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_15 2024-04-23T21:43:31.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_15 '' 2024-04-23T21:43:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_15 2024-04-23T21:43:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_15.status 2024-04-23T21:43:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_15.status 2024-04-23T21:43:31.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:31.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.status 2024-04-23T21:43:31.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_15 2024-04-23T21:43:31.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.status 2024-04-23T21:43:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:43:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:43:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15571], [object_number=14, tag_tid=1, entry_tid=15570]]], state=connected] 2024-04-23T21:43:31.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=15572], [object_number=15, tag_tid=1, entry_tid=15571], [object_number=14, tag_tid=1, entry_tid=15570], [object_number=13, tag_tid=1, entry_tid=15569]]], state=connected] 2024-04-23T21:43:31.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.status 2024-04-23T21:43:31.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:43:31.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_15 2024-04-23T21:43:31.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:31.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:31.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_15 2024-04-23T21:43:31.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_15 '..*' 2024-04-23T21:43:31.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:31.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:31.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_15 2024-04-23T21:43:31.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:31.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_15.status 2024-04-23T21:43:31.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_15.status 2024-04-23T21:43:31.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:31.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.status 2024-04-23T21:43:31.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_15 2024-04-23T21:43:31.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.status 2024-04-23T21:43:31.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:43:31.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:43:31.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:31.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15571], [object_number=14, tag_tid=1, entry_tid=15570]]], state=connected] 2024-04-23T21:43:31.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=15572], [object_number=15, tag_tid=1, entry_tid=15571], [object_number=14, tag_tid=1, entry_tid=15570], [object_number=13, tag_tid=1, entry_tid=15569]]], state=connected] 2024-04-23T21:43:31.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.status 2024-04-23T21:43:31.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=12, tag_tid=1, entry_tid=15572' 2024-04-23T21:43:31.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=12, tag_tid=1, entry_tid=15572' 2024-04-23T21:43:31.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=12, tag_tid=1, entry_tid=15572' '!=' '' 2024-04-23T21:43:31.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:43:31.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=12, tag_tid=1, entry_tid=15572' = '' 2024-04-23T21:43:31.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=12, tag_tid=1, entry_tid=15572' 2024-04-23T21:43:31.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:43:31.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:43:31.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:31.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:43:31.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=12, tag_tid=1, entry_tid=15572' 2024-04-23T21:43:31.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:43:31.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:31.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:43:31.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:43:31.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:43:31.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:31.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:43:31.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=12, tag_tid=1, entry_tid=15572' 2024-04-23T21:43:31.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:43:31.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:31.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=15572 2024-04-23T21:43:31.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 15572 2024-04-23T21:43:31.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:43:31.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:31.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:43:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_15 2024-04-23T21:43:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:43:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_15 ']' 2024-04-23T21:43:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_15' 2024-04-23T21:43:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_15 2024-04-23T21:43:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:31.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:31.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_15 2024-04-23T21:43:31.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:31.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:31.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:31.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:31.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:31.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:31.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:31.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:31.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:31.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:31.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:31.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:31.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:31.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:31.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:31.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:31.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:31.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:31.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:31.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:31.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:31.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:31.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:31.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:31.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:31.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:31.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:31.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:31.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:31.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:31.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:31.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:31.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_15 2024-04-23T21:43:31.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_15 2024-04-23T21:43:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_15 2024-04-23T21:43:31.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_15 '' 2024-04-23T21:43:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_15 2024-04-23T21:43:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_15.status 2024-04-23T21:43:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_15.status 2024-04-23T21:43:31.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:31.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.status 2024-04-23T21:43:31.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_15 2024-04-23T21:43:31.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.status 2024-04-23T21:43:32.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:43:32.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:43:32.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:32.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15571], [object_number=14, tag_tid=1, entry_tid=15570]]], state=connected] 2024-04-23T21:43:32.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15571], [object_number=14, tag_tid=1, entry_tid=15570]]], state=connected] 2024-04-23T21:43:32.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.status 2024-04-23T21:43:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:43:32.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_15 2024-04-23T21:43:32.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:32.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:32.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_15 2024-04-23T21:43:32.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_15 '..*' 2024-04-23T21:43:32.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:32.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:32.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_15 2024-04-23T21:43:32.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:32.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_15.status 2024-04-23T21:43:32.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_15.status 2024-04-23T21:43:32.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:32.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.status 2024-04-23T21:43:32.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_15 2024-04-23T21:43:32.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.status 2024-04-23T21:43:32.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:43:32.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:43:32.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:32.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15571], [object_number=14, tag_tid=1, entry_tid=15570]]], state=connected] 2024-04-23T21:43:32.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=15571], [object_number=14, tag_tid=1, entry_tid=15570]]], state=connected] 2024-04-23T21:43:32.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.status 2024-04-23T21:43:32.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:43:32.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:43:32.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:32.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_15 snap 2024-04-23T21:43:32.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:32.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:32.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:43:32.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:32.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:43:32.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:43:32.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:43:33.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_15@snap 2024-04-23T21:43:33.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_15': 2024-04-23T21:43:33.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:43:33.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:43:33.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:43:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 11b81080494a 2024-04-23T21:43:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.11b81080494a 2024-04-23T21:43:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:43:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:43:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:43:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:43:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:34:23 2024 2024-04-23T21:43:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:34:23 2024 2024-04-23T21:43:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:36:25 2024 2024-04-23T21:43:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:43:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 11b81080494a 2024-04-23T21:43:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:43:33.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:43:33.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: cd904466-1c61-4f0b-869a-10525a3dcdf0 2024-04-23T21:43:33.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:43:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_15 snap 2024-04-23T21:43:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:43:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:43:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.export 2024-04-23T21:43:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_15.export 2024-04-23T21:43:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_15.export 2024-04-23T21:43:33.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_15@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.export 2024-04-23T21:43:33.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:43:33.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_15@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_15.export 2024-04-23T21:43:33.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:43:33.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_15.export 2024-04-23T21:43:33.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_15.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_15.export 2024-04-23T21:43:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:43:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_16 2024-04-23T21:43:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_16 snap 2024-04-23T21:43:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:43:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:33.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:43:33.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:33.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_16@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:43:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_16 2024-04-23T21:43:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:43:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_16 started 2024-04-23T21:43:34.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:34.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:34.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:43:34.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:43:34.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:43:34.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:34.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:43:35.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_16 started 2024-04-23T21:43:35.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:35.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:35.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:43:35.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:43:35.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:43:35.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:43:35.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_16 2024-04-23T21:43:35.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:43:35.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:43:35.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:43:35.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:35.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:43:35.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:43:35.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:35.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:43:35.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:35.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:43:35.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:35.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_16 2024-04-23T21:43:35.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:43:35.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:43:35.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:35.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:35.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:35.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:43:35.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:35.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:35.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:35.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:35.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:43:35.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:43:35.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:35.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:35.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:35.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:43:35.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:43:35.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:43:35.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:35.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:43:35.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:43:35.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:43:35.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:43:35.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:35.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:35.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:43:35.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:43:35.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:43:35.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:43:35.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:35.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:35.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:35.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:35.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_16 2024-04-23T21:43:35.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_16'] 2024-04-23T21:43:35.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:43:35.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:35.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:35.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_16 2024-04-23T21:43:35.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:43:35.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:43:35.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:35.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:43:35.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:43:35.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:43:35.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:35.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:35.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:35.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:35.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:43:35.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:43:35.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:43:35.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:35.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:43:35.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:43:35.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:43:35.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:43:35.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:43:35.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:43:35.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:43:35.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:43:35.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:43:35.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:35.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:35.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:43:35.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:43:35.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:43:35.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:43:35.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:43:35.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:35.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:35.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:35.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_16 2024-04-23T21:43:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:43:35.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:43:35.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:43:35.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:43:35.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:43:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:43:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:43:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_16 2024-04-23T21:43:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:43:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:43:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:43:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:43:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_16 2024-04-23T21:43:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:43:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:43:35.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:35.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:43:35.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:43:35.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:43:35.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:43:35.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:35.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:43:35.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_16 2024-04-23T21:43:35.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:35.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:35.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:43:35.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:35.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_16 ']' 2024-04-23T21:43:35.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_16' 2024-04-23T21:43:35.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_16 2024-04-23T21:43:35.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:35.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:35.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:35.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:35.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:35.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:35.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:35.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:35.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:35.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:35.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_16 2024-04-23T21:43:35.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:35.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:35.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:35.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:35.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:35.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:35.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:35.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:35.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:35.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:35.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:35.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:35.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:35.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:35.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:35.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:35.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:35.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:35.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:35.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:35.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:35.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:35.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:35.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:35.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:35.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:35.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:35.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:35.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:35.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:35.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:35.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:35.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:35.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_16 2024-04-23T21:43:35.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_16'] 2024-04-23T21:43:35.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:35.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:35.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:35.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_16 2024-04-23T21:43:35.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:43:35.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:35.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:35.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:43:35.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:43:35.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:35.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:35.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:35.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:43:35.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:35.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:35.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:35.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:43:35.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:35.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:43:35.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:43:35.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:35.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:35.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:43:35.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:43:35.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:35.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:35.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:35.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:35.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:35.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:35.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:35.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:43:35.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:35.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:43:35.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:35.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:35.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:35.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_16 2024-04-23T21:43:35.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:35.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_16 2024-04-23T21:43:35.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:35.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:35.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:35.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_16 '' 2024-04-23T21:43:35.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:35.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:35.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:35.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:35.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_16.status 2024-04-23T21:43:35.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_16.status 2024-04-23T21:43:35.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:35.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:35.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_16 2024-04-23T21:43:35.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:35.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:43:35.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:43:35.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:35.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:35.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:35.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:35.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_16 2024-04-23T21:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_16 '..*' 2024-04-23T21:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:35.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_16.status 2024-04-23T21:43:35.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_16.status 2024-04-23T21:43:35.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:35.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:35.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_16 2024-04-23T21:43:35.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:35.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:43:35.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:43:35.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:35.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:35.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:35.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:35.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:35.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' '' 2024-04-23T21:43:35.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:43:35.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' = '' 2024-04-23T21:43:35.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:35.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:43:35.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:43:35.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:35.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:43:35.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:35.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:43:35.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:35.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:43:35.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:43:35.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:43:35.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:35.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:43:35.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:35.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:43:35.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=1 2024-04-23T21:43:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 1 2024-04-23T21:43:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:43:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:43:36.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_16 2024-04-23T21:43:36.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:36.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:36.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:43:36.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:36.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_16 ']' 2024-04-23T21:43:36.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_16' 2024-04-23T21:43:36.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_16 2024-04-23T21:43:36.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:36.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:36.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:36.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:36.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:36.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:36.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:36.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:36.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:36.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:36.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_16 2024-04-23T21:43:36.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:36.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:36.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:36.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:36.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:36.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:36.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:36.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:36.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:36.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_16 2024-04-23T21:43:36.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_16'] 2024-04-23T21:43:36.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:36.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:36.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:36.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_16 2024-04-23T21:43:36.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:43:36.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:36.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:36.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:43:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:43:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:43:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:43:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:43:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:43:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:36.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:43:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:43:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:43:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:43:36.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:36.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:36.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:36.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_16 2024-04-23T21:43:36.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:36.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_16 2024-04-23T21:43:36.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:36.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:36.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:36.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_16 '' 2024-04-23T21:43:36.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:36.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:36.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:36.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:36.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_16.status 2024-04-23T21:43:36.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_16.status 2024-04-23T21:43:36.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:36.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:36.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_16 2024-04-23T21:43:36.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:36.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:43:36.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:43:36.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:36.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:36.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:36.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:36.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:43:36.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_16 2024-04-23T21:43:36.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:36.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:36.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:36.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_16 '..*' 2024-04-23T21:43:36.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:36.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:36.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:36.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:36.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_16.status 2024-04-23T21:43:36.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_16.status 2024-04-23T21:43:36.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:36.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:36.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_16 2024-04-23T21:43:36.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:36.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:43:36.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:43:36.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:36.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:36.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:36.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:43:36.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_16 2024-04-23T21:43:36.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:36.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:36.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:43:36.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:36.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_16 ']' 2024-04-23T21:43:36.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_16' 2024-04-23T21:43:36.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_16 2024-04-23T21:43:36.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:36.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:36.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:36.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:36.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:36.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:36.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:36.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:36.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:36.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:36.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_16 2024-04-23T21:43:36.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:36.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:36.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:36.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:36.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:36.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:36.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:36.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:36.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:36.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:36.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:36.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:36.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:36.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:36.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:36.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:36.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:36.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:36.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_16 2024-04-23T21:43:36.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_16'] 2024-04-23T21:43:36.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:36.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_16 2024-04-23T21:43:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:43:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:43:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:43:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:43:36.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:43:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:43:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:43:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:43:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:43:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:36.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:36.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:36.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:43:36.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:36.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:43:36.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:36.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:36.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:36.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_16 2024-04-23T21:43:36.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:36.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_16 2024-04-23T21:43:36.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:36.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:36.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:36.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_16 '' 2024-04-23T21:43:36.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:36.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:36.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:36.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:36.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_16.status 2024-04-23T21:43:36.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_16.status 2024-04-23T21:43:36.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:36.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:36.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_16 2024-04-23T21:43:36.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:37.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:43:37.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:43:37.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:37.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:37.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:37.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:37.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:43:37.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_16 2024-04-23T21:43:37.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:37.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:37.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:37.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_16 '..*' 2024-04-23T21:43:37.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:37.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:37.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:37.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:37.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_16.status 2024-04-23T21:43:37.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_16.status 2024-04-23T21:43:37.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:37.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:37.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_16 2024-04-23T21:43:37.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:43:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:43:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:37.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:37.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:37.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:37.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:37.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:43:37.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_16 2024-04-23T21:43:37.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:37.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:37.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:43:37.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:37.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_16 ']' 2024-04-23T21:43:37.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_16' 2024-04-23T21:43:37.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_16 2024-04-23T21:43:37.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:37.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:37.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:37.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:37.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:37.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:37.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:37.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:37.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:37.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:37.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_16 2024-04-23T21:43:37.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:37.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:37.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:37.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:37.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:37.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:37.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:37.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:37.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:37.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:37.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:37.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:37.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:37.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:37.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:37.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:37.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:37.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:37.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:37.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:37.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:37.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:37.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:37.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:37.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:37.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:37.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:37.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:37.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:37.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_16 2024-04-23T21:43:37.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_16'] 2024-04-23T21:43:37.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:37.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:37.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:37.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_16 2024-04-23T21:43:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:43:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:43:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:43:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:43:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:43:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:37.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:43:37.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:43:37.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:37.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:37.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:43:37.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:43:37.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:37.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:37.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:37.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:37.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:37.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:37.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:37.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:43:37.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:37.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:43:38.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:38.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:38.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:38.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_16 2024-04-23T21:43:38.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:38.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_16 2024-04-23T21:43:38.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:38.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:38.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:38.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_16 '' 2024-04-23T21:43:38.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:38.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:38.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:38.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:38.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_16.status 2024-04-23T21:43:38.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_16.status 2024-04-23T21:43:38.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:38.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:38.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_16 2024-04-23T21:43:38.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:38.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:43:38.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:43:38.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:38.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:38.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:38.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:38.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:43:38.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_16 2024-04-23T21:43:38.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:38.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:38.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:38.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_16 '..*' 2024-04-23T21:43:38.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:38.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:38.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:38.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:38.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_16.status 2024-04-23T21:43:38.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_16.status 2024-04-23T21:43:38.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:38.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:38.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_16 2024-04-23T21:43:38.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:38.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:43:38.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:43:38.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:38.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:38.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:38.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:38.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:38.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:38.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:38.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:38.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:43:39.286 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8401381256733003) > weight (0.25) 2024-04-23T21:43:39.286 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8167011150777177) > weight (0.25) 2024-04-23T21:43:39.286 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9855067852629785) > weight (0.25) 2024-04-23T21:43:39.287 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6450336520747777) > weight (0.25) 2024-04-23T21:43:39.287 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2024-04-23T21:43:39.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_16 2024-04-23T21:43:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:43:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_16 ']' 2024-04-23T21:43:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_16' 2024-04-23T21:43:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_16 2024-04-23T21:43:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:39.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:39.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:39.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:39.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:39.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_16 2024-04-23T21:43:39.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:39.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:39.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:39.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:39.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:39.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:39.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:39.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:39.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:39.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:39.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:39.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:39.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:39.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:39.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:39.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:39.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:39.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:39.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:39.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_16 2024-04-23T21:43:39.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_16'] 2024-04-23T21:43:39.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:39.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:39.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:39.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_16 2024-04-23T21:43:39.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:43:39.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:39.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:39.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:43:39.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:43:39.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:39.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:39.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:39.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:43:39.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:39.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:39.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:39.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:43:39.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:39.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:43:39.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:43:39.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:39.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:39.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:43:39.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:43:39.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:43:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:43:39.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:39.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:39.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:39.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_16 2024-04-23T21:43:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_16 2024-04-23T21:43:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_16 '' 2024-04-23T21:43:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_16.status 2024-04-23T21:43:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_16.status 2024-04-23T21:43:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_16 2024-04-23T21:43:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:40.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:43:40.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:43:40.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:40.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:40.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:40.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:40.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:43:40.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_16 2024-04-23T21:43:40.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:40.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:40.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:40.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_16 '..*' 2024-04-23T21:43:40.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:40.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:40.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:40.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:40.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_16.status 2024-04-23T21:43:40.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_16.status 2024-04-23T21:43:40.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:40.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:40.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_16 2024-04-23T21:43:40.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:40.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:43:40.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:43:40.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:40.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:40.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:40.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:40.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:40.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:40.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:40.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:40.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:43:41.886 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3836077278423692) > weight (0.25) 2024-04-23T21:43:41.886 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3138104531584862) > weight (0.25) 2024-04-23T21:43:41.886 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.49225199035141176) > weight (0.25) 2024-04-23T21:43:41.886 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2024-04-23T21:43:41.886 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2024-04-23T21:43:41.887 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2024-04-23T21:43:41.887 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:43:41.888+0000 7f3305a2a640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 67088) UID: 1000 2024-04-23T21:43:42.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_16 2024-04-23T21:43:42.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:42.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:42.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:43:42.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:42.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_16 ']' 2024-04-23T21:43:42.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_16' 2024-04-23T21:43:42.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_16 2024-04-23T21:43:42.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:42.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:42.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:42.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:42.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:42.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:42.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:42.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:42.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:42.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_16 2024-04-23T21:43:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:42.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:42.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:42.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:42.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:42.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:42.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:42.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:42.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:42.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:42.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:42.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:42.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:42.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:42.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:42.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:42.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:42.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:42.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:42.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:42.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:42.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_16 2024-04-23T21:43:42.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_16'] 2024-04-23T21:43:42.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:42.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:42.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:42.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_16 2024-04-23T21:43:42.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:43:42.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:42.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:42.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:43:42.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:43:42.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:42.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:42.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:42.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:43:42.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:43:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:43:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:43:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:43:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:43:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:42.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:42.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:42.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:42.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:43:42.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:42.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:43:42.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:42.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:42.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:42.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_16 2024-04-23T21:43:42.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:42.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_16 2024-04-23T21:43:42.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:42.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:42.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:42.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_16 '' 2024-04-23T21:43:42.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:42.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:42.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:42.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:42.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_16.status 2024-04-23T21:43:42.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_16.status 2024-04-23T21:43:42.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:42.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:42.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_16 2024-04-23T21:43:42.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:42.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:43:42.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:43:42.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:42.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:42.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:42.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:42.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:43:42.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_16 2024-04-23T21:43:42.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:42.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:42.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:42.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_16 '..*' 2024-04-23T21:43:42.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:42.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:42.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:42.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:42.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_16.status 2024-04-23T21:43:42.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_16.status 2024-04-23T21:43:42.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:42.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:42.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_16 2024-04-23T21:43:42.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:43:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:43:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:42.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:42.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:42.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:42.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:42.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:42.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:43:42.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:42.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:43:44.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_16 2024-04-23T21:43:44.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:44.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:44.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:43:44.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:44.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_16 ']' 2024-04-23T21:43:44.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_16' 2024-04-23T21:43:44.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_16 2024-04-23T21:43:44.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:44.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:44.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:44.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:44.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:44.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:44.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:44.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:44.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:44.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:44.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_16 2024-04-23T21:43:44.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:44.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:44.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:44.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:44.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:44.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:44.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:44.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:44.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:44.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:44.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:44.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:44.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:44.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:44.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:44.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:44.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:44.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:44.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:44.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:44.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:44.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:44.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:44.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:44.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:44.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:44.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:44.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:44.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:44.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:44.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:44.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:44.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:44.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_16 2024-04-23T21:43:44.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_16'] 2024-04-23T21:43:44.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:43:44.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:43:44.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_16 2024-04-23T21:43:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:43:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:43:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:43:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:43:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:43:44.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:43:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:43:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:43:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:43:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:43:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:43:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:44.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:43:44.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:43:44.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:43:44.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:44.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:44.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:43:44.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_16 2024-04-23T21:43:44.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:44.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_16 2024-04-23T21:43:44.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:44.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:44.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:44.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_16 '' 2024-04-23T21:43:44.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:44.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:44.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:44.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:44.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_16.status 2024-04-23T21:43:44.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:44.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_16.status 2024-04-23T21:43:44.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:44.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_16 2024-04-23T21:43:44.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:44.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:43:44.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:43:44.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:44.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:44.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:44.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:44.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:43:44.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_16 2024-04-23T21:43:44.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:44.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:44.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:44.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_16 '..*' 2024-04-23T21:43:44.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:44.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:44.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:43:44.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:44.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_16.status 2024-04-23T21:43:44.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_16.status 2024-04-23T21:43:44.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:44.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:44.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_16 2024-04-23T21:43:44.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:44.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:43:44.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:43:44.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:44.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:44.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7263], [object_number=6, tag_tid=1, entry_tid=7262]]], state=connected] 2024-04-23T21:43:44.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.status 2024-04-23T21:43:44.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:43:44.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:43:44.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:44.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_16 snap 2024-04-23T21:43:44.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:44.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:44.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:43:44.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:44.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:43:44.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:43:44.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:43:45.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_16@snap 2024-04-23T21:43:45.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_16': 2024-04-23T21:43:45.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:43:45.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:43:45.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:43:45.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 120f7dc23bb6 2024-04-23T21:43:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.120f7dc23bb6 2024-04-23T21:43:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:43:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:43:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:43:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:43:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:34:37 2024 2024-04-23T21:43:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:34:37 2024 2024-04-23T21:43:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:36:53 2024 2024-04-23T21:43:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:43:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 120f7dc23bb6 2024-04-23T21:43:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:43:45.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:43:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: beb8b893-50aa-4213-b67f-f5999b5e53a2 2024-04-23T21:43:45.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:43:45.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:45.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_16 snap 2024-04-23T21:43:45.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:45.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:43:45.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:45.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:43:45.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.export 2024-04-23T21:43:45.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_16.export 2024-04-23T21:43:45.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_16.export 2024-04-23T21:43:45.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_16@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.export 2024-04-23T21:43:45.888 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2024-04-23T21:43:45.888 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:43:45.888 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2024-04-23T21:43:45.889 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2024-04-23T21:43:45.889 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2024-04-23T21:43:45.893 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2024-04-23T21:43:45.893 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2024-04-23T21:43:45.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:43:45.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_16@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_16.export 2024-04-23T21:43:46.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:43:46.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_16.export 2024-04-23T21:43:46.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_16.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_16.export 2024-04-23T21:43:46.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:46.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:43:46.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_17 2024-04-23T21:43:46.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_17 snap 2024-04-23T21:43:46.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:43:46.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:46.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:43:46.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:46.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_17@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:43:47.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_17 2024-04-23T21:43:47.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:47.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:47.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:43:47.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_17 started 2024-04-23T21:43:47.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:47.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:47.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:43:47.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:43:47.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:43:47.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:47.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:43:48.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_17 started 2024-04-23T21:43:48.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:48.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:48.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:43:48.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:43:48.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:43:48.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:43:48.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_17 2024-04-23T21:43:48.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:43:48.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:43:48.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:43:48.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:48.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:43:48.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:43:48.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:48.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:43:48.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:48.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:43:48.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:48.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_17 2024-04-23T21:43:48.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:43:48.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:43:48.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:48.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:48.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:48.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:43:48.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:48.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:48.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:48.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:48.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:43:48.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:43:48.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:48.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:48.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:48.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:43:48.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:43:48.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:43:48.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:48.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:43:48.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:43:48.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:43:48.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:43:48.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:48.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:48.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:43:48.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:43:48.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:43:48.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:43:48.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:48.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:48.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:48.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:48.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_17 2024-04-23T21:43:48.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:43:48.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:43:48.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:43:48.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:43:48.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:43:48.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:43:48.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:43:48.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:48.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_17 2024-04-23T21:43:48.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:43:48.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:43:48.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:48.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:43:48.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:43:48.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_17 2024-04-23T21:43:48.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:43:48.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:43:48.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:48.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:43:48.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:43:48.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:43:48.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:43:48.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:48.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:43:48.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_17 2024-04-23T21:43:48.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:48.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:48.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:43:48.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:48.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_17 ']' 2024-04-23T21:43:48.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_17' 2024-04-23T21:43:48.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_17 2024-04-23T21:43:48.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:48.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:48.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:48.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:48.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:48.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:48.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:48.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:48.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:48.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:48.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_17 2024-04-23T21:43:48.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:48.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:48.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:48.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:48.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:48.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:48.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:48.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:48.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:48.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:48.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:48.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:48.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:48.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:48.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:48.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:48.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_17 2024-04-23T21:43:48.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:48.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_17 2024-04-23T21:43:48.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:48.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:48.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:48.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_17 '' 2024-04-23T21:43:48.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:48.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:48.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:48.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:48.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_17.status 2024-04-23T21:43:48.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_17.status 2024-04-23T21:43:48.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:48.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:48.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_17 2024-04-23T21:43:48.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:48.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:43:48.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:43:48.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:48.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:48.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:48.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:48.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:43:48.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_17 2024-04-23T21:43:48.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:48.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:48.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:48.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_17 '..*' 2024-04-23T21:43:48.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:48.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:48.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:48.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:48.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_17.status 2024-04-23T21:43:48.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_17.status 2024-04-23T21:43:48.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_17 2024-04-23T21:43:48.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:48.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:43:48.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:43:48.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:48.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:48.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:48.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=3, entry_tid=1' -a 'object_number=1, tag_tid=3, entry_tid=1' = 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:48.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=2, entry_tid=1' '!=' '' 2024-04-23T21:43:48.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:43:48.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=2, entry_tid=1' = '' 2024-04-23T21:43:48.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:48.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:43:48.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:43:48.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:48.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:43:48.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:48.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:43:48.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:48.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:43:48.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:43:48.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:43:48.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:48.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:43:48.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:48.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:43:48.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:43:48.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=1 2024-04-23T21:43:48.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 1 2024-04-23T21:43:48.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:43:48.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:48.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:43:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_17 2024-04-23T21:43:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:43:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_17 ']' 2024-04-23T21:43:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_17' 2024-04-23T21:43:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_17 2024-04-23T21:43:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:48.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:48.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:48.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:48.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:48.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:48.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:48.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:48.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:48.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_17 2024-04-23T21:43:48.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:48.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:48.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:48.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:48.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:48.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:48.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:48.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:48.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:48.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:48.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:48.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:48.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:48.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:48.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:48.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:48.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:48.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:48.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:48.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:48.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:48.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:48.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:48.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:48.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:48.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:48.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:48.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:48.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:48.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:48.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:48.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:48.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:48.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_17 2024-04-23T21:43:48.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:48.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_17 2024-04-23T21:43:48.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:48.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:48.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:48.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_17 '' 2024-04-23T21:43:48.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:48.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:48.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:48.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:48.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_17.status 2024-04-23T21:43:48.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_17.status 2024-04-23T21:43:48.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:48.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:48.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_17 2024-04-23T21:43:48.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:43:49.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:43:49.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:49.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:49.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:49.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:49.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:43:49.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_17 2024-04-23T21:43:49.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:49.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:49.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:49.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_17 '..*' 2024-04-23T21:43:49.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:49.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:49.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:49.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:49.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_17.status 2024-04-23T21:43:49.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_17.status 2024-04-23T21:43:49.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:49.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:49.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_17 2024-04-23T21:43:49.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:49.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:43:49.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:43:49.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:49.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:49.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:49.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=3, entry_tid=1' -a 'object_number=1, tag_tid=3, entry_tid=1' = 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=2, entry_tid=1' '!=' 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:49.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:43:49.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_17 2024-04-23T21:43:49.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:49.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:49.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:43:49.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:49.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_17 ']' 2024-04-23T21:43:49.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_17' 2024-04-23T21:43:49.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_17 2024-04-23T21:43:49.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:49.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:49.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:49.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:49.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:49.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:49.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:49.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:49.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:49.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:49.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_17 2024-04-23T21:43:49.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:49.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:49.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:49.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:49.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:49.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:49.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:49.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:49.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:49.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:49.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:49.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:49.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:49.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:49.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:49.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:49.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:49.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:49.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:49.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:49.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:49.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:49.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:49.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:49.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:49.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:49.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:49.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:49.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:49.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:49.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:49.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:49.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:49.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_17 2024-04-23T21:43:49.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:49.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_17 2024-04-23T21:43:49.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:49.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:49.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:49.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_17 '' 2024-04-23T21:43:49.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:49.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:49.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:49.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:49.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_17.status 2024-04-23T21:43:49.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_17.status 2024-04-23T21:43:49.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:49.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:49.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_17 2024-04-23T21:43:49.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:49.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:43:49.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:43:49.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:49.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:49.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:49.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:49.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:43:49.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_17 2024-04-23T21:43:49.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:49.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:49.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_17 '..*' 2024-04-23T21:43:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_17.status 2024-04-23T21:43:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_17.status 2024-04-23T21:43:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:49.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:49.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_17 2024-04-23T21:43:49.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:49.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:43:49.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:43:49.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:49.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:49.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:49.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:49.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=3, entry_tid=1' -a 'object_number=1, tag_tid=3, entry_tid=1' = 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:49.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=2, entry_tid=1' '!=' 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:49.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:49.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:43:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_17 2024-04-23T21:43:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:43:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_17 ']' 2024-04-23T21:43:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_17' 2024-04-23T21:43:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_17 2024-04-23T21:43:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:50.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:50.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:50.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_17 2024-04-23T21:43:50.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:50.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:50.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:50.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:50.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:50.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:50.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:50.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:50.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:50.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:50.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:50.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:50.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:50.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:50.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:50.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:50.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:50.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:50.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:50.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:50.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:50.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:50.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:50.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:50.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:50.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:50.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:50.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:50.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:50.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:50.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:50.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:50.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:50.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_17 2024-04-23T21:43:50.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:50.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_17 2024-04-23T21:43:50.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:50.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:50.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:50.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_17 '' 2024-04-23T21:43:50.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:50.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:50.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:50.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:50.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_17.status 2024-04-23T21:43:50.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_17.status 2024-04-23T21:43:50.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:50.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:50.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_17 2024-04-23T21:43:50.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:50.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:43:50.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:43:50.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:50.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:50.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:50.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:50.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:43:50.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_17 2024-04-23T21:43:50.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:50.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:50.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:50.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_17 '..*' 2024-04-23T21:43:50.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:50.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:50.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:50.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:50.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_17.status 2024-04-23T21:43:50.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_17.status 2024-04-23T21:43:50.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:50.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:50.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_17 2024-04-23T21:43:50.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:50.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:43:50.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:43:50.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:50.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:50.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:50.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:50.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:50.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=3, entry_tid=1' -a 'object_number=1, tag_tid=3, entry_tid=1' = 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:50.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=2, entry_tid=1' '!=' 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:50.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:50.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:43:52.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_17 2024-04-23T21:43:52.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:52.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:52.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:43:52.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:52.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_17 ']' 2024-04-23T21:43:52.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_17' 2024-04-23T21:43:52.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_17 2024-04-23T21:43:52.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:52.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:52.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:52.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:52.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:52.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:52.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:52.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:52.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_17 2024-04-23T21:43:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:52.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:52.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:52.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:52.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:52.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:52.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:52.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:52.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:52.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:52.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_17 2024-04-23T21:43:52.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:52.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_17 2024-04-23T21:43:52.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:52.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:52.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:52.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_17 '' 2024-04-23T21:43:52.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:52.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:52.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:52.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:52.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_17.status 2024-04-23T21:43:52.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_17.status 2024-04-23T21:43:52.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:52.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:52.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_17 2024-04-23T21:43:52.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:52.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:43:52.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:43:52.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:52.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:52.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:52.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:52.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:43:52.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_17 2024-04-23T21:43:52.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:52.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:52.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:52.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_17 '..*' 2024-04-23T21:43:52.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:52.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:52.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:52.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:52.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_17.status 2024-04-23T21:43:52.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_17.status 2024-04-23T21:43:52.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:52.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:52.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_17 2024-04-23T21:43:52.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:52.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:43:52.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:43:52.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:52.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:52.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:52.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:52.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:52.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=3, entry_tid=1' -a 'object_number=1, tag_tid=3, entry_tid=1' = 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:52.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=2, entry_tid=1' '!=' 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:52.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:52.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:43:54.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_17 2024-04-23T21:43:54.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:54.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:54.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:43:54.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:54.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_17 ']' 2024-04-23T21:43:54.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_17' 2024-04-23T21:43:54.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_17 2024-04-23T21:43:54.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:54.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:54.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:54.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:54.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:54.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:54.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:54.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:54.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:54.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:54.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_17 2024-04-23T21:43:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:54.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:54.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:54.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:54.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:54.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:54.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:54.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:54.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:54.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:54.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:54.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:54.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:54.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:54.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:54.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:54.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_17 2024-04-23T21:43:54.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:54.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_17 2024-04-23T21:43:54.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:54.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:54.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:54.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_17 '' 2024-04-23T21:43:54.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:54.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:54.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:54.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:54.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_17.status 2024-04-23T21:43:54.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_17.status 2024-04-23T21:43:54.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:54.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:54.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_17 2024-04-23T21:43:54.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:54.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:43:54.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:43:54.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:54.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:54.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:54.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:54.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:43:54.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_17 2024-04-23T21:43:54.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:54.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:54.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:54.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_17 '..*' 2024-04-23T21:43:54.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:54.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:54.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:54.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:54.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_17.status 2024-04-23T21:43:54.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_17.status 2024-04-23T21:43:54.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:54.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:54.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_17 2024-04-23T21:43:54.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:54.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:43:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:43:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:54.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:54.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:54.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:54.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=3, entry_tid=1' -a 'object_number=1, tag_tid=3, entry_tid=1' = 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:54.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=2, entry_tid=1' '!=' 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:43:54.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:43:54.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:43:56.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_17 2024-04-23T21:43:56.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:56.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:56.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:43:56.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:43:56.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_17 ']' 2024-04-23T21:43:56.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_17' 2024-04-23T21:43:56.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_17 2024-04-23T21:43:56.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:43:56.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:56.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:56.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:43:56.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:43:56.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:43:56.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:43:56.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:43:56.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:43:56.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:43:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_17 2024-04-23T21:43:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:43:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:43:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:43:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:43:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:43:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:43:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:43:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:43:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:43:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:43:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:56.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:43:56.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:56.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:43:56.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:43:56.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:43:56.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:43:56.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:43:56.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:43:56.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:43:56.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:43:56.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:43:56.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:43:56.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:43:56.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:43:56.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:43:56.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:43:56.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:43:56.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:43:56.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:56.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:56.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:43:56.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_17 2024-04-23T21:43:56.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:56.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_17 2024-04-23T21:43:56.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:56.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:56.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:56.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_17 '' 2024-04-23T21:43:56.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:56.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:56.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:56.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:43:56.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_17.status 2024-04-23T21:43:56.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_17.status 2024-04-23T21:43:56.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:56.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:56.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_17 2024-04-23T21:43:56.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:56.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:43:56.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:43:56.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:56.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:56.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:56.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:56.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:43:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_17 2024-04-23T21:43:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_17 '..*' 2024-04-23T21:43:56.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:43:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:43:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:43:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:43:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_17.status 2024-04-23T21:43:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_17.status 2024-04-23T21:43:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:43:56.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:56.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_17 2024-04-23T21:43:56.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:57.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:43:57.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:43:57.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:43:57.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:57.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3719], [object_number=2, tag_tid=1, entry_tid=3718]]], state=connected] 2024-04-23T21:43:57.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.status 2024-04-23T21:43:57.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:43:57.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=3, entry_tid=1' -a 'object_number=1, tag_tid=3, entry_tid=1' = 'object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:43:57.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:57.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_17 snap 2024-04-23T21:43:57.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:43:57.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:57.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:43:57.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:57.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:43:57.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:43:57.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:43:58.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_17@snap 2024-04-23T21:43:58.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_17': 2024-04-23T21:43:58.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:43:58.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:43:58.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:43:58.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10b5bbbd1feb 2024-04-23T21:43:58.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10b5bbbd1feb 2024-04-23T21:43:58.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:43:58.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:43:58.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:43:58.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:43:58.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:34:45 2024 2024-04-23T21:43:58.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:34:45 2024 2024-04-23T21:43:58.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:35:46 2024 2024-04-23T21:43:58.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:43:58.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10b5bbbd1feb 2024-04-23T21:43:58.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:43:58.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:43:58.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 392693d9-0967-4ef3-99d1-909edb3bfad1 2024-04-23T21:43:58.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:43:58.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:58.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_17 snap 2024-04-23T21:43:58.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:58.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:43:58.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:58.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:43:58.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.export 2024-04-23T21:43:58.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_17.export 2024-04-23T21:43:58.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_17.export 2024-04-23T21:43:58.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_17@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.export 2024-04-23T21:43:58.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:43:58.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_17@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_17.export 2024-04-23T21:43:58.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:43:58.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_17.export 2024-04-23T21:43:58.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_17.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_17.export 2024-04-23T21:43:58.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:43:58.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:43:58.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_18 2024-04-23T21:43:58.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_18 snap 2024-04-23T21:43:58.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:43:58.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:43:58.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:43:58.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:43:58.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_18@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:44:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_18 2024-04-23T21:44:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:44:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_18 started 2024-04-23T21:44:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:44:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:44:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:44:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:00.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:44:01.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_18 started 2024-04-23T21:44:01.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:01.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:01.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:44:01.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:44:01.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:44:01.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:44:01.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_18 2024-04-23T21:44:01.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:44:01.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:44:01.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:01.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:01.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:44:01.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:44:01.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:01.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:44:01.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:01.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:44:01.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:01.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_18 2024-04-23T21:44:01.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:44:01.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:01.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:01.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:01.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:01.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:44:01.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:01.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:01.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:01.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:01.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:44:01.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:01.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:01.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:01.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:01.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:44:01.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:44:01.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:44:01.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:01.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:44:01.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:44:01.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:44:01.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:01.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:01.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:01.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:44:01.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:44:01.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:44:01.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:44:01.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:01.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:01.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:01.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:01.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_18 2024-04-23T21:44:01.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:44:01.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:44:01.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:44:01.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:44:01.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:44:01.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:44:01.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:44:01.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:01.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_18 2024-04-23T21:44:01.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:44:01.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:01.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:01.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:44:01.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:44:01.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_18 2024-04-23T21:44:01.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:44:01.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:01.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:01.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:44:01.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:44:01.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:44:01.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:44:01.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:01.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:44:01.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_18 2024-04-23T21:44:01.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:01.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:01.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:44:01.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:01.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_18 ']' 2024-04-23T21:44:01.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_18' 2024-04-23T21:44:01.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_18 2024-04-23T21:44:01.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:01.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:01.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:01.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:01.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:01.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:01.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:01.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:01.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:01.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:01.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_18 2024-04-23T21:44:01.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:01.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:01.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:01.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:01.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:01.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:01.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:01.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:01.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:01.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:01.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:01.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:01.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:01.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:01.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:01.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:01.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:01.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:01.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:01.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:01.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_18 2024-04-23T21:44:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_18 2024-04-23T21:44:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_18 '' 2024-04-23T21:44:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:01.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:01.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:01.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:01.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_18.status 2024-04-23T21:44:01.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_18.status 2024-04-23T21:44:01.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:01.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:01.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_18 2024-04-23T21:44:01.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:01.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:44:01.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:44:01.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:01.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:01.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=14769], [object_number=12, tag_tid=1, entry_tid=14768], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:01.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:01.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:44:01.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_18 2024-04-23T21:44:01.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:01.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:01.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:01.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_18 '..*' 2024-04-23T21:44:01.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:01.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:01.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:01.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:01.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_18.status 2024-04-23T21:44:01.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_18.status 2024-04-23T21:44:01.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:01.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:01.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_18 2024-04-23T21:44:01.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:44:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:44:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:01.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:01.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=14769], [object_number=12, tag_tid=1, entry_tid=14768], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:01.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:01.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:01.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:01.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=13, tag_tid=1, entry_tid=14769' '!=' '' 2024-04-23T21:44:01.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:44:01.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=13, tag_tid=1, entry_tid=14769' = '' 2024-04-23T21:44:01.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:01.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:44:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:44:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:01.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:44:01.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:01.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:44:01.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:01.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:44:01.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:44:01.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:44:01.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:01.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:44:01.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:01.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:44:01.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:01.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=14769 2024-04-23T21:44:01.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 14769 2024-04-23T21:44:01.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:44:01.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:01.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:44:02.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_18 2024-04-23T21:44:02.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:02.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:02.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:44:02.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:02.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_18 ']' 2024-04-23T21:44:02.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_18' 2024-04-23T21:44:02.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_18 2024-04-23T21:44:02.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:02.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:02.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:02.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:02.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:02.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:02.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:02.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:02.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:02.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:02.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_18 2024-04-23T21:44:02.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:02.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:02.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:02.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:02.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:02.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:02.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:02.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:02.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:02.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:02.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:02.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:02.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:02.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:02.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:02.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:02.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:02.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:02.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:02.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:02.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:02.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:02.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:02.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:02.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:02.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:02.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:02.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:02.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:02.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:02.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:02.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:02.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:02.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_18 2024-04-23T21:44:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_18 2024-04-23T21:44:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_18 '' 2024-04-23T21:44:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:02.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_18.status 2024-04-23T21:44:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_18.status 2024-04-23T21:44:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_18 2024-04-23T21:44:02.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:02.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:44:02.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:44:02.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:02.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:02.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=14769], [object_number=12, tag_tid=1, entry_tid=14768], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:02.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:02.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:44:02.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_18 2024-04-23T21:44:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_18 '..*' 2024-04-23T21:44:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_18.status 2024-04-23T21:44:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_18.status 2024-04-23T21:44:02.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_18 2024-04-23T21:44:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:02.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:44:02.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:44:02.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:02.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:02.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=14769], [object_number=12, tag_tid=1, entry_tid=14768], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:02.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:02.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:02.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:02.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=13, tag_tid=1, entry_tid=14769' '!=' 'object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:02.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:02.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:44:02.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_18 2024-04-23T21:44:02.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:02.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:02.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:44:02.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:02.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_18 ']' 2024-04-23T21:44:02.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_18' 2024-04-23T21:44:02.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_18 2024-04-23T21:44:02.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:02.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:02.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:02.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:02.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:02.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:02.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:02.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:02.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:02.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_18 2024-04-23T21:44:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:02.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:02.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:02.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:02.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:02.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:02.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:02.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:02.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:02.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:02.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:02.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:02.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:02.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:02.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:02.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:02.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:02.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:02.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:02.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:02.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:02.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:02.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:02.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:02.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_18 2024-04-23T21:44:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_18 2024-04-23T21:44:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_18 '' 2024-04-23T21:44:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_18.status 2024-04-23T21:44:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_18.status 2024-04-23T21:44:02.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:02.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:02.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_18 2024-04-23T21:44:02.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:02.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:44:02.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:44:02.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:02.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:02.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=14769], [object_number=12, tag_tid=1, entry_tid=14768], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:02.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:02.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:44:02.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_18 2024-04-23T21:44:02.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:02.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:02.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:02.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_18 '..*' 2024-04-23T21:44:02.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:02.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:02.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:02.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:02.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_18.status 2024-04-23T21:44:02.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_18.status 2024-04-23T21:44:02.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:02.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:02.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_18 2024-04-23T21:44:02.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:02.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:44:02.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:44:02.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:02.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:02.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=14769], [object_number=12, tag_tid=1, entry_tid=14768], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:02.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:02.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:02.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:02.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=13, tag_tid=1, entry_tid=14769' '!=' 'object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:02.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:02.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:44:03.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_18 2024-04-23T21:44:03.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:03.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:03.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:44:03.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:03.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_18 ']' 2024-04-23T21:44:03.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_18' 2024-04-23T21:44:03.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_18 2024-04-23T21:44:03.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:03.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:03.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:03.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:03.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:03.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:03.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:03.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:03.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:03.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:03.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_18 2024-04-23T21:44:03.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:03.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:03.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:03.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:03.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:03.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:03.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_18 2024-04-23T21:44:03.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:03.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_18 2024-04-23T21:44:03.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:03.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:03.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:03.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_18 '' 2024-04-23T21:44:03.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:03.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:03.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:03.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:03.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_18.status 2024-04-23T21:44:03.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_18.status 2024-04-23T21:44:03.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:03.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:03.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_18 2024-04-23T21:44:03.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:03.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:44:03.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:44:03.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:03.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:03.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=14769], [object_number=12, tag_tid=1, entry_tid=14768], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:03.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:03.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:44:03.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_18 2024-04-23T21:44:03.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:03.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:03.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:03.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_18 '..*' 2024-04-23T21:44:03.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:03.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:03.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:03.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:03.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_18.status 2024-04-23T21:44:03.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_18.status 2024-04-23T21:44:03.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:03.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:03.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_18 2024-04-23T21:44:03.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:03.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:44:03.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:44:03.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:03.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:03.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=14769], [object_number=12, tag_tid=1, entry_tid=14768], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:03.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:03.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:03.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:03.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=13, tag_tid=1, entry_tid=14769' '!=' 'object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:03.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:03.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:44:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_18 2024-04-23T21:44:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:44:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_18 ']' 2024-04-23T21:44:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_18' 2024-04-23T21:44:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_18 2024-04-23T21:44:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:05.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:05.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:05.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:05.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:05.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_18 2024-04-23T21:44:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:05.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:05.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:05.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:05.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:05.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:05.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:05.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:05.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:05.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_18 2024-04-23T21:44:05.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:05.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_18 2024-04-23T21:44:05.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:05.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:05.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:05.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_18 '' 2024-04-23T21:44:05.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:05.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:05.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:05.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:05.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_18.status 2024-04-23T21:44:05.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_18.status 2024-04-23T21:44:05.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:05.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:05.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_18 2024-04-23T21:44:05.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:05.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:44:05.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:44:05.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:05.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:05.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=14769], [object_number=12, tag_tid=1, entry_tid=14768], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:44:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_18 2024-04-23T21:44:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:05.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_18 '..*' 2024-04-23T21:44:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_18.status 2024-04-23T21:44:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_18.status 2024-04-23T21:44:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_18 2024-04-23T21:44:05.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:05.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:44:05.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:44:05.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:05.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:05.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=14769], [object_number=12, tag_tid=1, entry_tid=14768], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:05.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:05.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:05.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:05.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=13, tag_tid=1, entry_tid=14769' '!=' 'object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:05.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:05.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:44:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_18 2024-04-23T21:44:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:44:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_18 ']' 2024-04-23T21:44:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_18' 2024-04-23T21:44:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_18 2024-04-23T21:44:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:07.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:07.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:07.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:07.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:07.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:07.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:07.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_18 2024-04-23T21:44:07.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:07.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:07.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:07.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:07.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:07.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:07.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:07.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:07.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:07.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:07.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:07.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:07.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:07.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:07.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:07.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:07.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:07.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:07.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_18 2024-04-23T21:44:07.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:07.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_18 2024-04-23T21:44:07.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:07.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:07.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:07.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_18 '' 2024-04-23T21:44:07.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:07.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:07.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:07.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:07.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_18.status 2024-04-23T21:44:07.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_18.status 2024-04-23T21:44:07.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:07.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:07.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_18 2024-04-23T21:44:07.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:07.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:44:07.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:44:07.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:07.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:07.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=14769], [object_number=12, tag_tid=1, entry_tid=14768], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:07.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:07.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:44:07.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_18 2024-04-23T21:44:07.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:07.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:07.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:07.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_18 '..*' 2024-04-23T21:44:07.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:07.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:07.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:07.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:07.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_18.status 2024-04-23T21:44:07.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_18.status 2024-04-23T21:44:07.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:07.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:07.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_18 2024-04-23T21:44:07.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:44:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:44:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=14769], [object_number=12, tag_tid=1, entry_tid=14768], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:07.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:07.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:07.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:07.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=13, tag_tid=1, entry_tid=14769' '!=' 'object_number=13, tag_tid=1, entry_tid=14769' 2024-04-23T21:44:07.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:07.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:44:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_18 2024-04-23T21:44:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:44:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_18 ']' 2024-04-23T21:44:09.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_18' 2024-04-23T21:44:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_18 2024-04-23T21:44:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:09.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_18 2024-04-23T21:44:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:09.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:09.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:09.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:09.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:09.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_18 2024-04-23T21:44:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_18 2024-04-23T21:44:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_18 '' 2024-04-23T21:44:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_18.status 2024-04-23T21:44:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_18.status 2024-04-23T21:44:10.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:10.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:10.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_18 2024-04-23T21:44:10.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:10.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:44:10.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:44:10.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:10.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:10.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:10.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:10.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:44:10.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_18 2024-04-23T21:44:10.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:10.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:10.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:10.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_18 '..*' 2024-04-23T21:44:10.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:10.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:10.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:44:10.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:10.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_18.status 2024-04-23T21:44:10.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_18.status 2024-04-23T21:44:10.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:10.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:10.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_18 2024-04-23T21:44:10.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:10.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:44:10.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:44:10.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:10.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:10.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14767], [object_number=14, tag_tid=1, entry_tid=14766]]], state=connected] 2024-04-23T21:44:10.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.status 2024-04-23T21:44:10.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:44:10.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:44:10.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:10.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_18 snap 2024-04-23T21:44:10.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:10.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:10.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:44:10.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:44:10.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:44:10.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:44:10.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:44:11.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_18@snap 2024-04-23T21:44:11.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_18': 2024-04-23T21:44:11.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:44:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:44:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:44:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 116ef314541d 2024-04-23T21:44:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.116ef314541d 2024-04-23T21:44:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:44:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:44:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:44:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:44:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:34:50 2024 2024-04-23T21:44:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:34:50 2024 2024-04-23T21:44:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:36:52 2024 2024-04-23T21:44:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:44:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 116ef314541d 2024-04-23T21:44:11.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:44:11.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:44:11.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 0c02789b-418f-470c-84cb-adb57b1b4382 2024-04-23T21:44:11.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:44:11.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:11.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_18 snap 2024-04-23T21:44:11.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:11.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:44:11.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:44:11.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:44:11.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.export 2024-04-23T21:44:11.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_18.export 2024-04-23T21:44:11.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_18.export 2024-04-23T21:44:11.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_18@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.export 2024-04-23T21:44:11.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:44:11.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_18@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_18.export 2024-04-23T21:44:11.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:44:11.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_18.export 2024-04-23T21:44:11.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_18.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_18.export 2024-04-23T21:44:11.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:11.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:44:11.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_19 2024-04-23T21:44:11.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_19 snap 2024-04-23T21:44:11.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:11.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:11.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:44:11.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:44:11.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_19@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:44:12.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_19 2024-04-23T21:44:12.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:12.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:12.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:44:12.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_19 started 2024-04-23T21:44:12.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:12.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:12.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:44:12.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:44:12.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:44:12.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:12.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:44:13.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_19 started 2024-04-23T21:44:13.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:13.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:13.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:44:13.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:44:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:44:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:44:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_19 2024-04-23T21:44:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:44:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:44:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:44:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:44:13.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:13.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:44:13.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:13.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:44:13.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:13.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_19 2024-04-23T21:44:13.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:44:13.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:13.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:13.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:13.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:13.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:44:13.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:13.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:13.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:13.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:13.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:44:13.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:13.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:13.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:44:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:44:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:44:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:44:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:44:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:44:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:13.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:13.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:13.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:44:13.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:44:13.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:44:13.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:44:13.356 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:13.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:13.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:13.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:13.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_19 2024-04-23T21:44:13.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:44:13.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:44:13.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:44:13.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:44:13.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:44:13.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:44:13.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:44:13.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:13.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_19 2024-04-23T21:44:13.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:44:13.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:13.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:13.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:44:13.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:44:13.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_19 2024-04-23T21:44:13.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:44:13.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:13.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:13.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:44:13.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:44:13.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:44:13.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:44:13.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:13.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:44:13.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_19 2024-04-23T21:44:13.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:13.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:13.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:44:13.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:13.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_19 ']' 2024-04-23T21:44:13.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_19' 2024-04-23T21:44:13.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_19 2024-04-23T21:44:13.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:13.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:13.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:13.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:13.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:13.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:13.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:13.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:13.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:13.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:13.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_19 2024-04-23T21:44:13.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:13.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:13.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:13.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:13.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:13.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:13.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:13.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:13.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:13.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:13.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:13.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:13.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:13.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:13.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:13.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:13.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:13.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:13.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:13.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:13.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:13.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:13.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:13.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:13.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:13.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:13.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:13.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:13.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:13.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:13.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:13.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:13.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:13.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_19 2024-04-23T21:44:13.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:13.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_19 2024-04-23T21:44:13.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:13.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:13.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:13.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_19 '' 2024-04-23T21:44:13.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:13.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:13.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:13.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:13.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_19.status 2024-04-23T21:44:13.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_19.status 2024-04-23T21:44:13.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:13.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:13.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_19 2024-04-23T21:44:13.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:13.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:13.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:13.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:13.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:44:13.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_19 2024-04-23T21:44:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_19 '..*' 2024-04-23T21:44:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_19.status 2024-04-23T21:44:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_19.status 2024-04-23T21:44:13.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:13.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:13.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_19 2024-04-23T21:44:13.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:13.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:13.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:13.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:13.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:13.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' '' 2024-04-23T21:44:13.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:44:13.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' = '' 2024-04-23T21:44:13.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:13.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:44:13.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:44:13.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:13.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:44:13.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:13.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:44:13.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:13.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:44:13.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:44:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:44:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:13.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:44:13.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:13.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:44:13.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:13.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=1 2024-04-23T21:44:13.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 1 2024-04-23T21:44:13.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:44:13.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:13.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:44:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_19 2024-04-23T21:44:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:44:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_19 ']' 2024-04-23T21:44:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_19' 2024-04-23T21:44:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_19 2024-04-23T21:44:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:14.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:14.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:14.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:14.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:14.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:14.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:14.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:14.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:14.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_19 2024-04-23T21:44:14.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:14.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:14.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:14.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:14.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:14.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:14.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:14.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:14.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:14.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:14.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:14.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:14.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:14.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:14.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:14.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:14.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:14.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:14.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:14.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:14.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:14.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:14.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:14.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:14.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:14.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:14.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:14.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:14.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:14.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:14.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:14.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:14.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:14.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_19 2024-04-23T21:44:14.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:14.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_19 2024-04-23T21:44:14.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:14.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:14.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:14.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_19 '' 2024-04-23T21:44:14.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:14.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:14.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:14.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:14.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_19.status 2024-04-23T21:44:14.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_19.status 2024-04-23T21:44:14.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:14.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:14.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_19 2024-04-23T21:44:14.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:14.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:14.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:14.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:14.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:14.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:14.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:14.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:44:14.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_19 2024-04-23T21:44:14.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:14.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:14.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_19 '..*' 2024-04-23T21:44:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_19.status 2024-04-23T21:44:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_19.status 2024-04-23T21:44:14.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_19 2024-04-23T21:44:14.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:14.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:14.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:14.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:14.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:14.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:14.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:14.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:14.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:14.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:14.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:14.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:44:14.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_19 2024-04-23T21:44:14.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:14.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:14.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:44:14.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:14.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_19 ']' 2024-04-23T21:44:14.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_19' 2024-04-23T21:44:14.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_19 2024-04-23T21:44:14.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:14.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:14.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:14.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:14.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:14.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:14.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:14.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:14.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:14.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:14.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_19 2024-04-23T21:44:14.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:14.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:14.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:14.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:14.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:14.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:14.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:14.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:14.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:14.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:14.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:14.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:14.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:14.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:14.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:14.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:14.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:14.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:14.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:14.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:14.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_19 2024-04-23T21:44:14.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:14.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_19 2024-04-23T21:44:14.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:14.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:14.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:14.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_19 '' 2024-04-23T21:44:14.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:14.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:14.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:14.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:14.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_19.status 2024-04-23T21:44:14.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_19.status 2024-04-23T21:44:14.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:14.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:14.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_19 2024-04-23T21:44:14.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:14.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:14.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:14.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:14.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:14.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:14.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:14.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:44:14.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_19 2024-04-23T21:44:14.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:14.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:14.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:14.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_19 '..*' 2024-04-23T21:44:14.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:14.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:14.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:14.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:14.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_19.status 2024-04-23T21:44:14.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_19.status 2024-04-23T21:44:14.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:14.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:14.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_19 2024-04-23T21:44:14.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:14.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:14.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:14.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:14.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:14.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:14.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:14.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:44:15.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_19 2024-04-23T21:44:15.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:15.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:15.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:44:15.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:15.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_19 ']' 2024-04-23T21:44:15.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_19' 2024-04-23T21:44:15.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_19 2024-04-23T21:44:15.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:15.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:15.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:15.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:15.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:15.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:15.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:15.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:15.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:15.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:15.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_19 2024-04-23T21:44:15.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:15.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:15.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:15.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:15.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:15.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:15.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:15.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:15.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:15.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:15.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:15.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:15.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:15.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:15.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:15.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:15.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:15.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:15.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:15.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:15.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:15.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:15.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:15.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:15.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:15.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:15.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:15.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:15.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:15.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:15.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:15.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:15.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:15.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_19 2024-04-23T21:44:15.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:15.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_19 2024-04-23T21:44:15.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:15.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:15.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:15.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_19 '' 2024-04-23T21:44:15.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:15.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:15.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:15.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:15.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_19.status 2024-04-23T21:44:15.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_19.status 2024-04-23T21:44:15.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:15.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:15.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_19 2024-04-23T21:44:15.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:15.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:15.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:44:15.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_19 2024-04-23T21:44:15.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:15.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:15.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:15.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_19 '..*' 2024-04-23T21:44:15.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:15.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:15.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:15.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:15.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_19.status 2024-04-23T21:44:15.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_19.status 2024-04-23T21:44:15.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:15.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:15.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_19 2024-04-23T21:44:15.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:16.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:16.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:16.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:16.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:16.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:16.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:16.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:16.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:16.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:16.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:16.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:44:17.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_19 2024-04-23T21:44:17.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:17.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:17.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:44:17.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:17.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_19 ']' 2024-04-23T21:44:17.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_19' 2024-04-23T21:44:17.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_19 2024-04-23T21:44:17.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:17.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:17.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:17.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:17.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:17.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:17.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:17.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:17.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:17.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:17.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_19 2024-04-23T21:44:17.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:17.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:17.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:17.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:17.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:17.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:17.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:17.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:17.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:17.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:17.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:17.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:17.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:17.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:17.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:17.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:17.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:17.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:17.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:17.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:17.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:17.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:17.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:17.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:17.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:17.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:17.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:17.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:17.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:17.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:17.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:17.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:17.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:17.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_19 2024-04-23T21:44:17.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:17.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_19 2024-04-23T21:44:17.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:17.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:17.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:17.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_19 '' 2024-04-23T21:44:17.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:17.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:17.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:17.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:17.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_19.status 2024-04-23T21:44:17.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_19.status 2024-04-23T21:44:17.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:17.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:17.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_19 2024-04-23T21:44:17.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:17.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:17.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:17.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:17.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:17.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:17.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:17.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:44:17.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_19 2024-04-23T21:44:17.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:17.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:17.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:17.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_19 '..*' 2024-04-23T21:44:17.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:17.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:17.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:17.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:17.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_19.status 2024-04-23T21:44:17.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_19.status 2024-04-23T21:44:17.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:17.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:17.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_19 2024-04-23T21:44:17.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:17.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:17.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:17.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:17.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:17.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:17.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:17.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:17.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:17.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:17.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:17.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:44:19.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_19 2024-04-23T21:44:19.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:19.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:19.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:44:19.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:19.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_19 ']' 2024-04-23T21:44:19.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_19' 2024-04-23T21:44:19.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_19 2024-04-23T21:44:19.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:19.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:19.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:19.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:19.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:19.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:19.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:19.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:19.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:19.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:19.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_19 2024-04-23T21:44:19.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:19.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:19.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:19.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:19.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:19.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:19.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:19.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:19.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:19.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:19.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:19.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:19.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:19.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:19.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:19.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:19.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:19.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_19 2024-04-23T21:44:20.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:20.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_19 2024-04-23T21:44:20.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:20.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:20.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:20.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_19 '' 2024-04-23T21:44:20.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:20.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:20.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:20.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:20.046 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_19.status 2024-04-23T21:44:20.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_19.status 2024-04-23T21:44:20.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:20.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:20.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_19 2024-04-23T21:44:20.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:20.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:20.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:20.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:20.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:20.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:20.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:20.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:44:20.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_19 2024-04-23T21:44:20.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:20.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:20.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:20.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_19 '..*' 2024-04-23T21:44:20.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:20.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:20.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:20.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:20.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_19.status 2024-04-23T21:44:20.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_19.status 2024-04-23T21:44:20.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:20.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:20.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_19 2024-04-23T21:44:20.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:20.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:20.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:20.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:20.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:20.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:20.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:20.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:20.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:20.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:20.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:20.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:44:22.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_19 2024-04-23T21:44:22.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:22.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:22.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:44:22.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:22.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_19 ']' 2024-04-23T21:44:22.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_19' 2024-04-23T21:44:22.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_19 2024-04-23T21:44:22.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:22.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:22.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:22.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:22.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:22.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:22.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_19 2024-04-23T21:44:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:22.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:22.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:22.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:22.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:22.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:22.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_19 2024-04-23T21:44:22.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:22.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_19 2024-04-23T21:44:22.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:22.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:22.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:22.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_19 '' 2024-04-23T21:44:22.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:22.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:22.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:22.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:22.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_19.status 2024-04-23T21:44:22.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_19.status 2024-04-23T21:44:22.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:22.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:22.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_19 2024-04-23T21:44:22.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:22.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:22.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:22.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:22.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:22.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:22.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:22.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:44:22.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_19 2024-04-23T21:44:22.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:22.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:22.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:22.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_19 '..*' 2024-04-23T21:44:22.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:22.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:22.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:44:22.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:22.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_19.status 2024-04-23T21:44:22.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_19.status 2024-04-23T21:44:22.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:22.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:22.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_19 2024-04-23T21:44:22.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:22.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:22.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:22.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:22.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:22.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=4543], [object_number=6, tag_tid=1, entry_tid=4542]]], state=connected] 2024-04-23T21:44:22.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.status 2024-04-23T21:44:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:44:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:44:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_19 snap 2024-04-23T21:44:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:44:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:44:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:44:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:44:22.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:44:23.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_19@snap 2024-04-23T21:44:23.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_19': 2024-04-23T21:44:23.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:44:23.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:44:23.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:44:23.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 11b85bfee1a1 2024-04-23T21:44:23.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.11b85bfee1a1 2024-04-23T21:44:23.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:44:23.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:44:23.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:44:23.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:44:23.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:35:04 2024 2024-04-23T21:44:23.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:35:04 2024 2024-04-23T21:44:23.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:36:05 2024 2024-04-23T21:44:23.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:44:23.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 11b85bfee1a1 2024-04-23T21:44:23.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:44:23.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:44:23.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 14139a1e-394f-4652-a9dd-d5448df983eb 2024-04-23T21:44:23.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:44:23.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:23.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_19 snap 2024-04-23T21:44:23.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:23.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:44:23.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:44:23.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:44:23.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.export 2024-04-23T21:44:23.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_19.export 2024-04-23T21:44:23.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_19.export 2024-04-23T21:44:23.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_19@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.export 2024-04-23T21:44:23.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:44:23.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_19@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_19.export 2024-04-23T21:44:23.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:44:23.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_19.export 2024-04-23T21:44:23.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_19.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_19.export 2024-04-23T21:44:23.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:23.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:44:23.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_20 2024-04-23T21:44:23.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_20 snap 2024-04-23T21:44:23.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:23.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:23.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:44:23.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:44:23.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_20@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:44:25.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_20 2024-04-23T21:44:25.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:25.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:25.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:44:25.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_20 started 2024-04-23T21:44:25.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:44:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:44:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:44:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:44:26.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_20 started 2024-04-23T21:44:26.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:26.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:26.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:44:26.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:44:26.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:44:26.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:44:26.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_20 2024-04-23T21:44:26.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:44:26.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:44:26.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:26.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:26.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:44:26.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:44:26.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:26.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:44:26.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:26.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:44:26.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:26.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_20 2024-04-23T21:44:26.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:44:26.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:26.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:26.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:26.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:26.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:44:26.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:26.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:26.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:26.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:26.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:44:26.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:26.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:26.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:26.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:26.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:44:26.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:44:26.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:44:26.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:26.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:44:26.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:44:26.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:44:26.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:26.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:26.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:26.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:44:26.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:44:26.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:44:26.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:44:26.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:26.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:26.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:26.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:26.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_20 2024-04-23T21:44:26.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_20'] 2024-04-23T21:44:26.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:44:26.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_20 2024-04-23T21:44:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:44:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:44:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:26.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:44:26.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:26.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:26.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:26.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:26.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:44:26.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:44:26.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:44:26.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:26.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:44:26.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:44:26.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:44:26.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:26.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:26.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:26.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:44:26.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:44:26.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:44:26.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:44:26.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:26.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:26.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:26.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:26.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_20 2024-04-23T21:44:26.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:44:26.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:44:26.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:44:26.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:44:26.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:44:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:44:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:44:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_20 2024-04-23T21:44:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:44:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:44:26.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:44:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_20 2024-04-23T21:44:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:44:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:44:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:44:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:44:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:44:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:26.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:44:26.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_20 2024-04-23T21:44:26.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:26.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:26.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:44:26.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:26.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_20 ']' 2024-04-23T21:44:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_20' 2024-04-23T21:44:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_20 2024-04-23T21:44:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:26.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:26.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:26.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:26.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_20 2024-04-23T21:44:26.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:26.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:26.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:26.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:26.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:26.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:26.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:26.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:26.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:26.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:26.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:26.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:26.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:26.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:26.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:26.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:26.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:26.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:26.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:26.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:26.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_20 2024-04-23T21:44:26.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_20'] 2024-04-23T21:44:26.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:26.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:26.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:26.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_20 2024-04-23T21:44:26.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:44:26.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:26.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:26.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:26.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:44:26.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:26.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:26.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:26.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:44:26.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:26.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:26.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:26.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:26.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:26.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:26.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:26.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:26.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_20 2024-04-23T21:44:27.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:27.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_20 2024-04-23T21:44:27.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:27.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:27.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:44:27.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_20 '' 2024-04-23T21:44:27.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:27.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:27.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:44:27.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:27.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_20.status 2024-04-23T21:44:27.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_20.status 2024-04-23T21:44:27.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:27.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:27.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_20 2024-04-23T21:44:27.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:27.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:27.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:27.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:27.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8111], [object_number=6, tag_tid=1, entry_tid=8110]]], state=connected] 2024-04-23T21:44:27.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=8112], [object_number=7, tag_tid=1, entry_tid=8111], [object_number=6, tag_tid=1, entry_tid=8110], [object_number=5, tag_tid=1, entry_tid=8109]]], state=connected] 2024-04-23T21:44:27.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:27.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:27.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_20 2024-04-23T21:44:27.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:27.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:27.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:44:27.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_20 '..*' 2024-04-23T21:44:27.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:27.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:27.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:44:27.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:27.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_20.status 2024-04-23T21:44:27.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_20.status 2024-04-23T21:44:27.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:27.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:27.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_20 2024-04-23T21:44:27.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:27.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:27.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:27.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:27.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8111], [object_number=6, tag_tid=1, entry_tid=8110]]], state=connected] 2024-04-23T21:44:27.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=8112], [object_number=7, tag_tid=1, entry_tid=8111], [object_number=6, tag_tid=1, entry_tid=8110], [object_number=5, tag_tid=1, entry_tid=8109]]], state=connected] 2024-04-23T21:44:27.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:27.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=4, tag_tid=1, entry_tid=8112' 2024-04-23T21:44:27.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=4, tag_tid=1, entry_tid=8112' 2024-04-23T21:44:27.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=4, tag_tid=1, entry_tid=8112' '!=' '' 2024-04-23T21:44:27.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:44:27.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=4, tag_tid=1, entry_tid=8112' = '' 2024-04-23T21:44:27.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=4, tag_tid=1, entry_tid=8112' 2024-04-23T21:44:27.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:27.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:44:27.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:27.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:44:27.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=4, tag_tid=1, entry_tid=8112' 2024-04-23T21:44:27.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:44:27.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:27.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:44:27.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:27.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:44:27.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:27.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:44:27.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=4, tag_tid=1, entry_tid=8112' 2024-04-23T21:44:27.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:44:27.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:27.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=8112 2024-04-23T21:44:27.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 8112 2024-04-23T21:44:27.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:44:27.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:27.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:44:27.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_20 2024-04-23T21:44:27.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:27.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:27.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:44:27.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:27.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_20 ']' 2024-04-23T21:44:27.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_20' 2024-04-23T21:44:27.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_20 2024-04-23T21:44:27.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:27.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:27.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:27.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:27.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:27.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:27.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:27.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:27.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:27.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:27.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_20 2024-04-23T21:44:27.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:27.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:27.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:27.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:27.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:27.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:27.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:27.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:27.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:27.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:27.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:27.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:27.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:27.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:27.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:27.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:27.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:27.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:27.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_20 2024-04-23T21:44:27.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_20'] 2024-04-23T21:44:27.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:27.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:27.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:27.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_20 2024-04-23T21:44:27.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:44:27.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:27.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:27.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:27.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:44:27.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:27.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:27.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:27.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:44:27.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:27.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:27.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:27.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:44:27.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:27.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:27.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:44:27.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:27.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:27.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:27.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:27.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:27.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:27.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:27.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:27.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:27.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:27.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:27.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:27.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:27.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:27.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:27.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:27.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:27.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_20 2024-04-23T21:44:27.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:27.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_20 2024-04-23T21:44:27.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:27.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:27.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:44:27.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_20 '' 2024-04-23T21:44:27.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:27.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:27.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:44:27.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:27.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_20.status 2024-04-23T21:44:27.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_20.status 2024-04-23T21:44:27.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:27.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:27.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_20 2024-04-23T21:44:27.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:27.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:27.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:27.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:27.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8111], [object_number=6, tag_tid=1, entry_tid=8110]]], state=connected] 2024-04-23T21:44:27.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=8112], [object_number=7, tag_tid=1, entry_tid=8111], [object_number=6, tag_tid=1, entry_tid=8110], [object_number=5, tag_tid=1, entry_tid=8109]]], state=connected] 2024-04-23T21:44:27.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:27.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:27.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_20 2024-04-23T21:44:27.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:27.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:27.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:44:27.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_20 '..*' 2024-04-23T21:44:27.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:27.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:27.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:44:27.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:27.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_20.status 2024-04-23T21:44:27.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_20.status 2024-04-23T21:44:27.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:27.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:27.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_20 2024-04-23T21:44:27.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:27.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:27.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:27.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:27.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8111], [object_number=6, tag_tid=1, entry_tid=8110]]], state=connected] 2024-04-23T21:44:27.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=8112], [object_number=7, tag_tid=1, entry_tid=8111], [object_number=6, tag_tid=1, entry_tid=8110], [object_number=5, tag_tid=1, entry_tid=8109]]], state=connected] 2024-04-23T21:44:27.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:27.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=4, tag_tid=1, entry_tid=8112' 2024-04-23T21:44:27.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=4, tag_tid=1, entry_tid=8112' 2024-04-23T21:44:27.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=4, tag_tid=1, entry_tid=8112' '!=' 'object_number=4, tag_tid=1, entry_tid=8112' 2024-04-23T21:44:27.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:27.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:44:28.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_20 2024-04-23T21:44:28.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:28.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:28.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:44:28.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:28.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_20 ']' 2024-04-23T21:44:28.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_20' 2024-04-23T21:44:28.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_20 2024-04-23T21:44:28.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:28.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:28.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:28.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:28.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:28.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:28.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:28.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:28.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:28.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:28.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_20 2024-04-23T21:44:28.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:28.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:28.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:28.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:28.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:28.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:28.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:28.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:28.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:28.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:28.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:28.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:28.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:28.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:28.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:28.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:28.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:28.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:28.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:28.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:28.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:28.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:28.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:28.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:28.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:28.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:28.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:28.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:28.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:28.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:28.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:28.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:28.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:28.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_20 2024-04-23T21:44:28.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_20'] 2024-04-23T21:44:28.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:28.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:28.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:28.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_20 2024-04-23T21:44:28.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:44:28.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:28.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:28.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:28.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:44:28.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:28.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:28.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:28.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:44:28.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:28.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:28.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:28.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:44:28.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:28.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:28.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:44:28.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:28.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:28.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:28.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:28.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:28.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:28.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:28.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:28.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:28.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:28.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:28.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:28.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:28.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:28.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:28.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:28.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:28.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_20 2024-04-23T21:44:28.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:28.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_20 2024-04-23T21:44:28.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:28.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:28.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:44:28.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_20 '' 2024-04-23T21:44:28.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:28.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:28.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:44:28.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:28.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_20.status 2024-04-23T21:44:28.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_20.status 2024-04-23T21:44:28.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:28.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:28.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_20 2024-04-23T21:44:28.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:28.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:28.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:28.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:28.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8111], [object_number=6, tag_tid=1, entry_tid=8110]]], state=connected] 2024-04-23T21:44:28.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=8112], [object_number=7, tag_tid=1, entry_tid=8111], [object_number=6, tag_tid=1, entry_tid=8110], [object_number=5, tag_tid=1, entry_tid=8109]]], state=connected] 2024-04-23T21:44:28.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:28.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:28.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_20 2024-04-23T21:44:28.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:28.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:28.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:44:28.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_20 '..*' 2024-04-23T21:44:28.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:28.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:28.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:44:28.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:28.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_20.status 2024-04-23T21:44:28.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_20.status 2024-04-23T21:44:28.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:28.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:28.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_20 2024-04-23T21:44:28.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:28.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:28.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:28.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:28.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8111], [object_number=6, tag_tid=1, entry_tid=8110]]], state=connected] 2024-04-23T21:44:28.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=8112], [object_number=7, tag_tid=1, entry_tid=8111], [object_number=6, tag_tid=1, entry_tid=8110], [object_number=5, tag_tid=1, entry_tid=8109]]], state=connected] 2024-04-23T21:44:28.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:28.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=4, tag_tid=1, entry_tid=8112' 2024-04-23T21:44:28.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=4, tag_tid=1, entry_tid=8112' 2024-04-23T21:44:28.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=4, tag_tid=1, entry_tid=8112' '!=' 'object_number=4, tag_tid=1, entry_tid=8112' 2024-04-23T21:44:28.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:28.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:44:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_20 2024-04-23T21:44:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:44:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_20 ']' 2024-04-23T21:44:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_20' 2024-04-23T21:44:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_20 2024-04-23T21:44:29.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:29.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:29.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:29.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:29.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:29.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:29.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:29.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:29.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:29.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:29.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_20 2024-04-23T21:44:29.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:29.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:29.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:29.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:29.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:29.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:29.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:29.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:29.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:29.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:29.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:29.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:29.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:29.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:29.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:29.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:29.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:29.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:29.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:29.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:29.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:29.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:29.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:29.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:29.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:29.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:29.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:29.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:29.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:29.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:29.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:29.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:29.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:29.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_20 2024-04-23T21:44:29.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_20'] 2024-04-23T21:44:29.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:29.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_20 2024-04-23T21:44:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:44:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:44:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:29.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:44:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:44:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:44:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:29.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:29.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:29.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:29.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:29.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:29.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:29.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:29.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:29.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:29.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:29.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:29.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_20 2024-04-23T21:44:29.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:29.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_20 2024-04-23T21:44:29.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:29.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:29.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:44:29.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_20 '' 2024-04-23T21:44:29.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:29.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:29.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:44:29.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:29.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_20.status 2024-04-23T21:44:29.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_20.status 2024-04-23T21:44:29.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:29.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:29.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_20 2024-04-23T21:44:29.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:29.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:29.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:29.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:29.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8111], [object_number=6, tag_tid=1, entry_tid=8110]]], state=connected] 2024-04-23T21:44:29.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8111], [object_number=6, tag_tid=1, entry_tid=8110]]], state=connected] 2024-04-23T21:44:29.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:29.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_20 2024-04-23T21:44:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:44:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_20 '..*' 2024-04-23T21:44:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:44:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_20.status 2024-04-23T21:44:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_20.status 2024-04-23T21:44:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:29.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:29.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_20 2024-04-23T21:44:29.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:29.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:29.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:29.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:29.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8111], [object_number=6, tag_tid=1, entry_tid=8110]]], state=connected] 2024-04-23T21:44:29.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=8111], [object_number=6, tag_tid=1, entry_tid=8110]]], state=connected] 2024-04-23T21:44:29.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.status 2024-04-23T21:44:29.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:29.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:29.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:29.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_20 snap 2024-04-23T21:44:29.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:29.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:29.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:44:29.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:44:29.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:44:29.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:44:29.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:44:30.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_20@snap 2024-04-23T21:44:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_20': 2024-04-23T21:44:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:44:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:44:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:44:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 120f766970da 2024-04-23T21:44:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.120f766970da 2024-04-23T21:44:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:44:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:44:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:44:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:44:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:35:08 2024 2024-04-23T21:44:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:35:08 2024 2024-04-23T21:44:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:36:09 2024 2024-04-23T21:44:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:44:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 120f766970da 2024-04-23T21:44:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:44:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:44:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 1661c736-9249-4795-b17d-50a964cad861 2024-04-23T21:44:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:44:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_20 snap 2024-04-23T21:44:30.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:44:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:44:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:44:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.export 2024-04-23T21:44:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_20.export 2024-04-23T21:44:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_20.export 2024-04-23T21:44:30.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_20@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.export 2024-04-23T21:44:30.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:44:30.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_20@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_20.export 2024-04-23T21:44:31.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:44:31.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_20.export 2024-04-23T21:44:31.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_20.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_20.export 2024-04-23T21:44:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:44:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_21 2024-04-23T21:44:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_21 snap 2024-04-23T21:44:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:44:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:44:31.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_21@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:44:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_21 2024-04-23T21:44:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:44:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_21 started 2024-04-23T21:44:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:44:32.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:44:32.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:44:32.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:32.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:44:33.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_21 started 2024-04-23T21:44:33.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:33.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:33.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:44:33.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:44:33.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:44:33.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:44:33.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_21 2024-04-23T21:44:33.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:44:33.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:44:33.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:33.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:33.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:44:33.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:44:33.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:33.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:44:33.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:33.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:44:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_21 2024-04-23T21:44:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:44:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:44:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:33.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:44:33.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:33.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:33.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:33.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:33.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:44:33.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:44:33.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:44:33.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:33.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:44:33.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:44:33.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:44:33.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:33.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:33.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:33.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:44:33.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:44:33.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:44:33.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:44:33.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:33.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:33.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:33.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:33.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_21 2024-04-23T21:44:33.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:33.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:44:33.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:33.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:33.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_21 2024-04-23T21:44:33.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:44:33.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:33.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:33.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:44:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:44:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:44:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:44:33.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:44:33.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:33.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:44:33.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:44:33.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:44:33.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:33.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:33.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:33.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:44:33.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:44:33.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:44:33.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:44:33.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:33.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:33.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:33.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:33.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_21 2024-04-23T21:44:33.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:33.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:44:33.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:33.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:33.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_21 2024-04-23T21:44:33.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:44:33.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:33.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:33.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:44:33.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:44:33.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:44:33.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:33.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:33.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:33.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:33.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:44:33.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:33.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:44:33.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:44:33.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:44:33.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:44:33.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:44:33.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:44:33.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:44:33.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:44:33.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:44:33.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:44:33.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:33.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:44:33.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:33.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:44:33.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:44:33.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:44:33.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:44:33.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:44:33.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:33.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:33.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:33.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_21 2024-04-23T21:44:33.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:33.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:44:33.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:33.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:33.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_21 2024-04-23T21:44:33.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:44:33.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:33.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:33.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:44:33.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:44:33.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:44:33.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:33.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:33.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:33.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:33.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:44:33.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:33.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:44:33.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:44:33.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:44:33.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:44:33.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:44:33.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:44:33.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:44:33.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:44:33.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:44:33.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:44:33.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:33.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:44:33.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:33.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:44:33.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:44:33.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:44:33.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:44:33.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:44:33.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:33.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:33.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:33.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_21 2024-04-23T21:44:33.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:44:33.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:44:33.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:44:33.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:44:33.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:44:33.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:44:33.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:44:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_21 2024-04-23T21:44:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:44:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:44:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:44:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_21 2024-04-23T21:44:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:44:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:44:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:44:33.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:44:33.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:44:33.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:33.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:44:34.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_21 2024-04-23T21:44:34.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:34.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:34.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:44:34.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:34.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_21 ']' 2024-04-23T21:44:34.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_21' 2024-04-23T21:44:34.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_21 2024-04-23T21:44:34.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:34.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:34.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:34.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:34.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:34.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:34.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:34.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:34.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:34.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_21 2024-04-23T21:44:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:34.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:34.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:34.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:34.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:34.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:34.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:34.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:34.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:34.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:34.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:34.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:34.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:34.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:34.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:34.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:34.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:34.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:34.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:34.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:34.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:34.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:34.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:34.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:34.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:34.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:34.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:34.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:34.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:34.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_21 2024-04-23T21:44:34.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:44:34.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:34.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:34.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:34.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:44:34.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:34.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:44:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:44:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:44:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:34.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:34.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:34.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:34.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:34.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:34.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:34.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:34.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:34.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:34.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:34.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:34.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:34.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:34.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:34.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:34.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:34.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:34.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_21 2024-04-23T21:44:34.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:44:34.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:34.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:34.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:44:34.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:44:34.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:34.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:34.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:34.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:44:34.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:34.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:34.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:34.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:44:34.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:34.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:44:34.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:44:34.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:34.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:34.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:44:34.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:44:34.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:34.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:34.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:34.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:34.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:34.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:34.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:34.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:44:34.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:44:34.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:44:34.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:34.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:34.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:34.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:34.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:34.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:34.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:34.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:34.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_21 2024-04-23T21:44:34.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:44:34.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:34.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:34.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:44:34.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:44:34.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:34.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:34.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:44:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:44:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:44:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:44:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:44:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:44:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:34.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:34.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:34.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:34.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:34.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:44:34.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:44:34.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:44:34.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:34.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:34.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:34.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_21 2024-04-23T21:44:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_21 '' 2024-04-23T21:44:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:34.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:34.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:34.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_21.status 2024-04-23T21:44:34.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_21.status 2024-04-23T21:44:34.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:34.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:34.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_21 2024-04-23T21:44:34.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:44:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:44:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=1, entry_tid=4029], [object_number=0, tag_tid=1, entry_tid=4028], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:34.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:44:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_21 2024-04-23T21:44:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_21 '..*' 2024-04-23T21:44:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_21.status 2024-04-23T21:44:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_21.status 2024-04-23T21:44:34.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:34.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:34.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_21 2024-04-23T21:44:34.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:34.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:44:34.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:44:34.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:34.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:34.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=1, entry_tid=4029], [object_number=0, tag_tid=1, entry_tid=4028], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:34.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:34.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=1, entry_tid=4029' 2024-04-23T21:44:34.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=2, entry_tid=1' -a 'object_number=1, tag_tid=2, entry_tid=1' = 'object_number=1, tag_tid=1, entry_tid=4029' 2024-04-23T21:44:34.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=1, entry_tid=4029' '!=' '' 2024-04-23T21:44:34.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:44:34.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=1, entry_tid=4029' = '' 2024-04-23T21:44:34.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=1, tag_tid=1, entry_tid=4029' 2024-04-23T21:44:34.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:44:34.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:44:34.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:34.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:44:34.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=1, tag_tid=1, entry_tid=4029' 2024-04-23T21:44:34.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:44:34.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:34.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:44:34.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:44:34.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:44:34.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:34.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:44:34.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=1, tag_tid=1, entry_tid=4029' 2024-04-23T21:44:34.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:44:34.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:34.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=4029 2024-04-23T21:44:34.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 4029 2024-04-23T21:44:34.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:44:34.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:34.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:44:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_21 2024-04-23T21:44:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:44:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_21 ']' 2024-04-23T21:44:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_21' 2024-04-23T21:44:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_21 2024-04-23T21:44:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:34.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:34.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_21 2024-04-23T21:44:34.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:34.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:34.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:34.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:34.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:34.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:34.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:34.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:34.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:34.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:34.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:34.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:34.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:34.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:34.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:34.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:34.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:34.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:34.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:34.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:34.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:34.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:34.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:34.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_21 2024-04-23T21:44:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:44:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:44:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:44:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:34.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:34.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:34.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:34.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:34.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:34.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:34.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:34.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:34.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:34.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_21 2024-04-23T21:44:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:44:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:44:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:44:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:44:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:34.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:44:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:44:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:44:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:44:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:44:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:34.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:44:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:44:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:44:34.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:34.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:34.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:34.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:35.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:35.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:35.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_21 2024-04-23T21:44:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:44:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:44:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:44:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:44:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:35.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:44:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:44:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:44:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:44:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:44:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:35.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:35.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:35.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:44:35.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:44:35.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:44:35.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:35.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:35.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:35.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_21 2024-04-23T21:44:35.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:35.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:35.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:35.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_21 '' 2024-04-23T21:44:35.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:35.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:35.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:35.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:35.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_21.status 2024-04-23T21:44:35.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_21.status 2024-04-23T21:44:35.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:35.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:35.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_21 2024-04-23T21:44:35.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:35.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:44:35.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:44:35.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:35.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:35.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=1, entry_tid=4029], [object_number=0, tag_tid=1, entry_tid=4028], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:35.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:35.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:44:35.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_21 2024-04-23T21:44:35.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:35.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:35.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:35.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_21 '..*' 2024-04-23T21:44:35.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:35.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:35.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:35.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:35.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_21.status 2024-04-23T21:44:35.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_21.status 2024-04-23T21:44:35.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:35.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:35.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_21 2024-04-23T21:44:35.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:35.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:44:35.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:44:35.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:35.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:35.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=1, entry_tid=4029], [object_number=0, tag_tid=1, entry_tid=4028], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:35.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:35.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=1, entry_tid=4029' 2024-04-23T21:44:35.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=2, entry_tid=1' -a 'object_number=1, tag_tid=2, entry_tid=1' = 'object_number=1, tag_tid=1, entry_tid=4029' 2024-04-23T21:44:35.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=1, entry_tid=4029' '!=' 'object_number=1, tag_tid=1, entry_tid=4029' 2024-04-23T21:44:35.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:35.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:44:35.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_21 2024-04-23T21:44:35.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:35.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:35.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:44:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_21 ']' 2024-04-23T21:44:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_21' 2024-04-23T21:44:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_21 2024-04-23T21:44:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:35.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:35.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:35.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_21 2024-04-23T21:44:35.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:35.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:35.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:35.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:35.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:35.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:35.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:35.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:35.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:35.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:35.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:35.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:35.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:35.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:35.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:35.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:35.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:35.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:35.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:35.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:35.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:35.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:35.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_21 2024-04-23T21:44:35.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:44:35.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:35.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:35.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:35.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:44:35.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:35.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:44:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:44:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:44:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:35.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:35.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:35.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:35.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:36.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:36.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:36.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_21 2024-04-23T21:44:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:44:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:44:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:44:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:44:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:36.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:44:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:44:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:44:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:44:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:44:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:36.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:36.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:36.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:44:36.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:44:36.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:44:36.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:36.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:36.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:36.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:36.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:36.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:36.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:36.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_21 2024-04-23T21:44:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:44:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:44:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:44:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:44:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:36.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:44:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:44:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:44:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:44:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:44:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:36.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:44:36.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:44:36.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:44:36.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:36.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:36.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:36.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:36.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:36.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_21 2024-04-23T21:44:36.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:36.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:36.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:36.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_21 '' 2024-04-23T21:44:36.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:36.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:36.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:36.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:36.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_21.status 2024-04-23T21:44:36.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_21.status 2024-04-23T21:44:36.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:36.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:36.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_21 2024-04-23T21:44:36.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:36.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:44:36.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:44:36.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:36.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:36.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=1, entry_tid=4029], [object_number=0, tag_tid=1, entry_tid=4028], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:36.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:36.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:44:36.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_21 2024-04-23T21:44:36.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:36.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:36.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:36.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_21 '..*' 2024-04-23T21:44:36.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:36.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:36.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:36.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:36.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_21.status 2024-04-23T21:44:36.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_21.status 2024-04-23T21:44:36.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:36.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:36.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_21 2024-04-23T21:44:36.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:44:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:44:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=1, entry_tid=4029], [object_number=0, tag_tid=1, entry_tid=4028], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:36.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:36.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=1, entry_tid=4029' 2024-04-23T21:44:36.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=2, entry_tid=1' -a 'object_number=1, tag_tid=2, entry_tid=1' = 'object_number=1, tag_tid=1, entry_tid=4029' 2024-04-23T21:44:36.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=1, entry_tid=4029' '!=' 'object_number=1, tag_tid=1, entry_tid=4029' 2024-04-23T21:44:36.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:36.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:44:37.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_21 2024-04-23T21:44:37.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:37.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:37.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:44:37.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:37.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_21 ']' 2024-04-23T21:44:37.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_21' 2024-04-23T21:44:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_21 2024-04-23T21:44:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:37.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:37.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_21 2024-04-23T21:44:37.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:37.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:37.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:37.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:37.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:37.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:37.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:37.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:37.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:37.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:37.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:37.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:37.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:37.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:37.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:37.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:37.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:37.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:37.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:37.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:37.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:37.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:37.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:37.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:37.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:37.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:37.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:37.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:37.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:37.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:37.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:37.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:37.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:37.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:37.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:37.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_21 2024-04-23T21:44:37.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:44:37.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:37.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:37.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:37.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:44:37.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:37.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:37.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:37.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:44:37.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:37.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:37.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:37.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:44:37.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:37.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:37.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:44:37.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:37.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:37.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:37.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:37.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:37.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:37.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:37.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:37.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:37.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:37.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:37.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:37.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:37.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:37.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:37.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:37.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:37.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:37.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:37.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:37.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:37.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:37.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_21 2024-04-23T21:44:37.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:44:37.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:37.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:37.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:44:37.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:44:37.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:37.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:37.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:37.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:44:37.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:37.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:37.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:37.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:44:37.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:37.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:44:37.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:44:37.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:37.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:37.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:44:37.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:44:37.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:37.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:37.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:37.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:37.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:37.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:37.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:37.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:44:37.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:44:37.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:44:37.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:37.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:37.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:37.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:37.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:37.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:37.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:37.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:37.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_21 2024-04-23T21:44:37.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:44:37.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:37.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:37.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:44:37.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:44:37.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:37.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:37.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:37.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:44:37.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:37.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:37.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:37.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:44:37.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:44:37.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:37.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:37.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:37.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:37.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:37.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_21 2024-04-23T21:44:37.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:37.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:37.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:37.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_21 '' 2024-04-23T21:44:37.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:37.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:37.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:37.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:37.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_21.status 2024-04-23T21:44:37.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_21.status 2024-04-23T21:44:37.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:37.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:37.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_21 2024-04-23T21:44:37.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:37.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:44:37.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:44:37.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:37.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:37.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=1, entry_tid=4029], [object_number=0, tag_tid=1, entry_tid=4028], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:37.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:37.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:44:37.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_21 2024-04-23T21:44:37.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:37.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:37.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:37.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_21 '..*' 2024-04-23T21:44:37.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:37.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:37.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:37.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:37.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_21.status 2024-04-23T21:44:37.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_21.status 2024-04-23T21:44:37.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:37.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:37.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_21 2024-04-23T21:44:37.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:37.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:44:37.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:44:37.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:37.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:37.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=1, entry_tid=4029], [object_number=0, tag_tid=1, entry_tid=4028], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:37.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:37.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=1, entry_tid=4029' 2024-04-23T21:44:37.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=2, entry_tid=1' -a 'object_number=1, tag_tid=2, entry_tid=1' = 'object_number=1, tag_tid=1, entry_tid=4029' 2024-04-23T21:44:37.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=1, entry_tid=4029' '!=' 'object_number=1, tag_tid=1, entry_tid=4029' 2024-04-23T21:44:37.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:37.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:44:39.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_21 2024-04-23T21:44:39.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:39.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:39.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:44:39.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:39.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_21 ']' 2024-04-23T21:44:39.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_21' 2024-04-23T21:44:39.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_21 2024-04-23T21:44:39.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:39.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:39.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:39.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:39.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:39.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:39.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_21 2024-04-23T21:44:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:39.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:39.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:39.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:39.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:39.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:39.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:39.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:39.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:39.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:39.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:39.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_21 2024-04-23T21:44:39.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:44:39.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:39.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:39.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:39.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:44:39.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:39.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:39.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:39.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:44:39.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:39.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:39.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:39.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:44:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:44:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:39.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:39.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:39.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:39.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:39.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:39.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:39.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:39.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:39.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:39.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:39.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_21 2024-04-23T21:44:39.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:44:39.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:39.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:39.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:44:39.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:44:39.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:39.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:39.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:39.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:44:39.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:39.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:39.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:39.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:44:39.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:39.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:44:39.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:44:39.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:39.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:39.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:44:39.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:44:39.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:39.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:39.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:39.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:39.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:39.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:39.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:39.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:44:39.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:44:39.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:44:39.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:39.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:39.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:39.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:39.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_21'] 2024-04-23T21:44:39.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:39.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:39.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_21 2024-04-23T21:44:39.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:44:39.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:39.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:39.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:44:39.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:44:39.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:39.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:44:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:44:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:44:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:44:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:44:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:44:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:44:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:44:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:44:39.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:39.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:39.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:39.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_21 2024-04-23T21:44:39.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:39.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_21 2024-04-23T21:44:39.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:39.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:39.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:39.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_21 '' 2024-04-23T21:44:39.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:39.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:39.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:39.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:39.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_21.status 2024-04-23T21:44:39.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_21.status 2024-04-23T21:44:39.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:39.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:39.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_21 2024-04-23T21:44:39.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:39.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:44:39.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:44:39.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:39.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:39.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:39.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:39.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:44:39.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_21 2024-04-23T21:44:39.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:39.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:39.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:39.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_21 '..*' 2024-04-23T21:44:39.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:39.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:39.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:44:39.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:39.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_21.status 2024-04-23T21:44:39.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_21.status 2024-04-23T21:44:39.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:39.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:39.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_21 2024-04-23T21:44:39.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:39.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:44:39.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:44:39.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:39.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:39.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=4027], [object_number=2, tag_tid=1, entry_tid=4026]]], state=connected] 2024-04-23T21:44:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.status 2024-04-23T21:44:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:44:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=2, entry_tid=1' -a 'object_number=1, tag_tid=2, entry_tid=1' = 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:44:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_21 snap 2024-04-23T21:44:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:44:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:44:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:44:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:44:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:44:40.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_21@snap 2024-04-23T21:44:40.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_21': 2024-04-23T21:44:40.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:44:40.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:44:40.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:44:40.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10b59b79c028 2024-04-23T21:44:40.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10b59b79c028 2024-04-23T21:44:40.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:44:40.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:44:40.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:44:40.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:44:40.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:35:16 2024 2024-04-23T21:44:40.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:35:16 2024 2024-04-23T21:44:40.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:35:16 2024 2024-04-23T21:44:40.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:44:40.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10b59b79c028 2024-04-23T21:44:40.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:44:40.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:44:40.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 1b75ece5-811c-4d25-a3d1-0f1254cacc38 2024-04-23T21:44:40.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:44:40.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:40.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_21 snap 2024-04-23T21:44:40.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:40.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:44:40.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:44:40.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:44:40.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.export 2024-04-23T21:44:40.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_21.export 2024-04-23T21:44:40.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_21.export 2024-04-23T21:44:40.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_21@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.export 2024-04-23T21:44:40.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:44:41.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_21@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_21.export 2024-04-23T21:44:41.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:44:41.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_21.export 2024-04-23T21:44:41.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_21.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_21.export 2024-04-23T21:44:41.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:41.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:44:41.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_22 2024-04-23T21:44:41.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_22 snap 2024-04-23T21:44:41.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:41.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:41.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:44:41.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:44:41.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_22@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:44:42.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_22 2024-04-23T21:44:42.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:42.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:42.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:44:42.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_22 started 2024-04-23T21:44:42.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:42.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:42.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:44:42.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:44:42.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:44:42.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:42.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:44:43.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_22 started 2024-04-23T21:44:43.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:43.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:43.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:44:43.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:44:43.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:44:43.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:44:43.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_22 2024-04-23T21:44:43.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:44:43.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:44:43.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:43.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:43.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:44:43.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:44:43.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:43.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:44:43.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:43.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:44:43.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:43.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_22 2024-04-23T21:44:43.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:44:43.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:43.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:43.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:43.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:43.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:44:43.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:43.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:43.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:43.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:43.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:44:43.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:43.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:43.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:43.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:43.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:44:43.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:44:43.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:44:43.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:43.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:44:43.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:44:43.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:44:43.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:43.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:43.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:43.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:44:43.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:44:43.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:44:43.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:44:43.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:43.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:43.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:43.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:43.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_22 2024-04-23T21:44:43.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:44:43.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:44:43.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:44:43.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:44:43.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:44:43.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:44:43.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:44:43.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:43.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_22 2024-04-23T21:44:43.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:44:43.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:43.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:43.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:44:43.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:44:43.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_22 2024-04-23T21:44:43.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:44:43.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:43.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:43.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:44:43.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:44:43.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:44:43.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:44:43.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:43.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:44:43.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_22 2024-04-23T21:44:43.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:43.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:43.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:44:43.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:43.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_22 ']' 2024-04-23T21:44:43.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_22' 2024-04-23T21:44:43.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_22 2024-04-23T21:44:43.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:43.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:43.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:43.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:43.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:43.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:43.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:43.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:43.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:43.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:43.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_22 2024-04-23T21:44:43.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:43.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:43.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:43.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:43.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:43.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:43.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:43.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:43.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:43.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:43.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:43.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:43.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:43.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:43.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:43.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:43.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:43.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:43.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:43.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:43.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_22 2024-04-23T21:44:43.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:43.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_22 2024-04-23T21:44:43.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:43.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:43.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_22 2024-04-23T21:44:43.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_22 '' 2024-04-23T21:44:43.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:43.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:43.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_22 2024-04-23T21:44:43.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:43.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_22.status 2024-04-23T21:44:43.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_22.status 2024-04-23T21:44:43.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:43.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.status 2024-04-23T21:44:43.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_22 2024-04-23T21:44:43.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.status 2024-04-23T21:44:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6979], [object_number=6, tag_tid=1, entry_tid=6978]]], state=connected] 2024-04-23T21:44:43.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=6980], [object_number=7, tag_tid=1, entry_tid=6979], [object_number=6, tag_tid=1, entry_tid=6978], [object_number=5, tag_tid=1, entry_tid=6977]]], state=connected] 2024-04-23T21:44:43.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.status 2024-04-23T21:44:43.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:43.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_22 2024-04-23T21:44:43.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:43.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:43.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_22 2024-04-23T21:44:43.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_22 '..*' 2024-04-23T21:44:43.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:43.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:43.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_22 2024-04-23T21:44:43.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:43.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_22.status 2024-04-23T21:44:43.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_22.status 2024-04-23T21:44:43.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:43.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.status 2024-04-23T21:44:43.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_22 2024-04-23T21:44:43.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.status 2024-04-23T21:44:43.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:43.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:43.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:43.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6979], [object_number=6, tag_tid=1, entry_tid=6978]]], state=connected] 2024-04-23T21:44:43.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=6980], [object_number=7, tag_tid=1, entry_tid=6979], [object_number=6, tag_tid=1, entry_tid=6978], [object_number=5, tag_tid=1, entry_tid=6977]]], state=connected] 2024-04-23T21:44:43.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.status 2024-04-23T21:44:43.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=4, tag_tid=1, entry_tid=6980' 2024-04-23T21:44:43.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=4, tag_tid=1, entry_tid=6980' 2024-04-23T21:44:43.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=4, tag_tid=1, entry_tid=6980' '!=' '' 2024-04-23T21:44:43.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:44:43.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=4, tag_tid=1, entry_tid=6980' = '' 2024-04-23T21:44:43.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=4, tag_tid=1, entry_tid=6980' 2024-04-23T21:44:43.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:43.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:44:43.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:43.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:44:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=4, tag_tid=1, entry_tid=6980' 2024-04-23T21:44:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:44:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:44:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:44:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:44:43.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=4, tag_tid=1, entry_tid=6980' 2024-04-23T21:44:43.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:44:43.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:43.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=6980 2024-04-23T21:44:43.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 6980 2024-04-23T21:44:43.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:44:43.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:43.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:44:44.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_22 2024-04-23T21:44:44.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:44.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:44.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:44:44.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:44.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_22 ']' 2024-04-23T21:44:44.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_22' 2024-04-23T21:44:44.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_22 2024-04-23T21:44:44.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:44.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:44.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:44.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:44.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:44.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:44.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:44.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:44.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_22 2024-04-23T21:44:44.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:44.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:44.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:44.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:44.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:44.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:44.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:44.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:44.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:44.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:44.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:44.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:44.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:44.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:44.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:44.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:44.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:44.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:44.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:44.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:44.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_22 2024-04-23T21:44:44.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:44.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_22 2024-04-23T21:44:44.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:44.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:44.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_22 2024-04-23T21:44:44.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_22 '' 2024-04-23T21:44:44.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:44.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:44.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_22 2024-04-23T21:44:44.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:44.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_22.status 2024-04-23T21:44:44.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_22.status 2024-04-23T21:44:44.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:44.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.status 2024-04-23T21:44:44.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_22 2024-04-23T21:44:44.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.status 2024-04-23T21:44:44.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:44.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:44.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:44.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6979], [object_number=6, tag_tid=1, entry_tid=6978]]], state=connected] 2024-04-23T21:44:44.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6979], [object_number=6, tag_tid=1, entry_tid=6978]]], state=connected] 2024-04-23T21:44:44.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.status 2024-04-23T21:44:44.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:44.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_22 2024-04-23T21:44:44.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:44.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:44.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_22 2024-04-23T21:44:44.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_22 '..*' 2024-04-23T21:44:44.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:44.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:44.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_22 2024-04-23T21:44:44.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:44.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_22.status 2024-04-23T21:44:44.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_22.status 2024-04-23T21:44:44.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:44.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.status 2024-04-23T21:44:44.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_22 2024-04-23T21:44:44.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.status 2024-04-23T21:44:44.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:44.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:44.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:44.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6979], [object_number=6, tag_tid=1, entry_tid=6978]]], state=connected] 2024-04-23T21:44:44.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6979], [object_number=6, tag_tid=1, entry_tid=6978]]], state=connected] 2024-04-23T21:44:44.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.status 2024-04-23T21:44:44.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:44.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:44.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:44.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_22 snap 2024-04-23T21:44:44.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:44.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:44.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:44:44.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:44:44.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:44:44.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:44:44.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:44:45.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_22@snap 2024-04-23T21:44:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_22': 2024-04-23T21:44:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:44:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:44:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:44:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 116e3aa8c1f 2024-04-23T21:44:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.116e3aa8c1f 2024-04-23T21:44:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:44:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:44:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:44:45.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:44:45.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:35:24 2024 2024-04-23T21:44:45.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:35:24 2024 2024-04-23T21:44:45.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:36:25 2024 2024-04-23T21:44:45.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:44:45.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 116e3aa8c1f 2024-04-23T21:44:45.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:44:45.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:44:45.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 0cef9c31-a3f7-4c03-8cd5-7acfc2466456 2024-04-23T21:44:45.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:44:45.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:45.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_22 snap 2024-04-23T21:44:45.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:45.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:44:45.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:44:45.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:44:45.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.export 2024-04-23T21:44:45.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_22.export 2024-04-23T21:44:45.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_22.export 2024-04-23T21:44:45.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_22@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.export 2024-04-23T21:44:45.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:44:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_22@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_22.export 2024-04-23T21:44:45.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:44:45.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_22.export 2024-04-23T21:44:45.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_22.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_22.export 2024-04-23T21:44:45.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:45.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:44:45.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_23 2024-04-23T21:44:45.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_23 snap 2024-04-23T21:44:45.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:45.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:45.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:44:45.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:44:45.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_23@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:44:46.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_23 2024-04-23T21:44:46.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:46.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:46.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:44:46.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_23 started 2024-04-23T21:44:46.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:46.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:46.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:44:46.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:44:46.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:44:46.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:46.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:44:47.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_23 started 2024-04-23T21:44:47.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:47.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:47.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:44:47.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:44:47.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:44:47.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:44:47.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_23 2024-04-23T21:44:47.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:44:47.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:44:47.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:47.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:47.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:44:47.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:44:47.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:47.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:44:47.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:47.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:44:47.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:47.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_23 2024-04-23T21:44:47.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:44:47.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:44:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:44:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:47.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:44:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:44:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:44:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:44:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:44:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:44:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:44:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:44:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:44:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:44:47.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:47.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:47.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:47.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:47.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_23 2024-04-23T21:44:47.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:47.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:44:47.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:47.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:47.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_23 2024-04-23T21:44:47.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:44:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:44:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:44:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:44:47.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:44:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:44:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:44:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:44:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:44:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:44:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:44:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:44:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:44:47.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:47.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:47.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:47.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:47.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_23 2024-04-23T21:44:47.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:47.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:44:47.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:47.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:47.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_23 2024-04-23T21:44:47.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:44:47.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:44:47.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:47.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:44:47.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:44:47.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:44:47.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:47.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:47.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:47.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:47.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:44:47.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:47.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:44:47.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:44:47.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:44:47.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:44:47.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:44:47.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:44:47.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:44:47.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:44:47.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:44:47.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:44:47.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:44:47.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:44:47.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:47.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:44:47.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:44:47.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:44:47.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:44:47.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:44:47.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:47.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:47.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:47.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_23 2024-04-23T21:44:47.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:44:47.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:44:47.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:44:47.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:44:47.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:44:47.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:44:47.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:44:47.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:47.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_23 2024-04-23T21:44:47.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:44:47.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:47.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:47.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:44:47.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:44:47.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_23 2024-04-23T21:44:47.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:44:47.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:44:47.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:47.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:44:47.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:44:47.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:44:47.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:44:47.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:47.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:44:47.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_23 2024-04-23T21:44:47.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:47.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:47.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:44:47.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:47.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_23 ']' 2024-04-23T21:44:47.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_23' 2024-04-23T21:44:47.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_23 2024-04-23T21:44:47.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:47.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:47.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:47.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:47.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:47.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:47.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:47.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:47.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:47.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:47.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_23 2024-04-23T21:44:47.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:47.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:47.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:47.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:47.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:47.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:47.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:47.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:47.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:47.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:47.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:47.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:47.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:47.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:47.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:47.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:47.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:47.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:47.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:47.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:47.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:47.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:47.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:47.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:47.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:47.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:47.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:47.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:47.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:47.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:47.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:47.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:47.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:47.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:48.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:48.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:48.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:48.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:48.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_23 2024-04-23T21:44:48.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:44:48.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:48.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:48.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:48.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:44:48.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:48.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:48.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:48.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:44:48.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:48.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:48.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:48.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:44:48.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:48.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:48.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:44:48.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:48.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:48.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:48.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:48.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:48.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:48.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:48.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:48.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:48.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:48.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:48.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:48.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:48.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:48.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:48.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:48.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:48.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:48.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:48.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:48.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:48.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:48.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_23 2024-04-23T21:44:48.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:44:48.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:44:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:44:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:44:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:44:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:44:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:44:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:48.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:44:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:44:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:44:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:44:48.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:44:48.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:48.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:48.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:48.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:48.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:48.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_23 2024-04-23T21:44:48.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:48.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:48.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:44:48.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_23 '' 2024-04-23T21:44:48.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:48.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:48.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:44:48.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:48.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_23.status 2024-04-23T21:44:48.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_23.status 2024-04-23T21:44:48.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:48.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:48.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_23 2024-04-23T21:44:48.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:48.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:48.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:48.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:48.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7067], [object_number=6, tag_tid=1, entry_tid=7066]]], state=connected] 2024-04-23T21:44:48.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7067], [object_number=6, tag_tid=1, entry_tid=7066]]], state=connected] 2024-04-23T21:44:48.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:48.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:44:48.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_23 2024-04-23T21:44:48.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:48.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:48.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:44:48.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_23 '..*' 2024-04-23T21:44:48.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:48.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:48.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:44:48.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:48.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_23.status 2024-04-23T21:44:48.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_23.status 2024-04-23T21:44:48.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:48.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:48.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_23 2024-04-23T21:44:48.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:48.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:48.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:48.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:48.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7067], [object_number=6, tag_tid=1, entry_tid=7066]]], state=connected] 2024-04-23T21:44:48.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7067], [object_number=6, tag_tid=1, entry_tid=7066]]], state=connected] 2024-04-23T21:44:48.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:48.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' '' 2024-04-23T21:44:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:44:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' = '' 2024-04-23T21:44:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:44:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:44:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:48.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:44:48.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:48.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:44:48.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:48.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:44:48.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:44:48.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:44:48.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:48.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:44:48.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:48.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:44:48.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:44:48.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=1 2024-04-23T21:44:48.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 1 2024-04-23T21:44:48.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:44:48.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:48.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:44:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_23 2024-04-23T21:44:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:44:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_23 ']' 2024-04-23T21:44:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_23' 2024-04-23T21:44:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_23 2024-04-23T21:44:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:48.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:48.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:48.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:48.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:48.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:48.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:48.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:48.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_23 2024-04-23T21:44:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:48.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:48.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:48.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:48.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:48.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:48.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:48.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:48.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:48.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:48.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:48.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_23 2024-04-23T21:44:48.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:44:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:44:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:44:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:44:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:48.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:44:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:48.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:48.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:48.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:48.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:48.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:48.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_23 2024-04-23T21:44:48.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:44:48.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:48.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:44:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:44:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:44:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:44:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:44:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:44:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:48.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:44:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:44:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:44:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:44:48.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:44:48.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:48.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:48.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:48.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:48.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:48.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_23 2024-04-23T21:44:48.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:48.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:48.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:44:48.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_23 '' 2024-04-23T21:44:48.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:48.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:48.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:44:48.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:48.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_23.status 2024-04-23T21:44:48.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_23.status 2024-04-23T21:44:48.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:48.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:48.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_23 2024-04-23T21:44:48.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:48.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:48.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7067], [object_number=6, tag_tid=1, entry_tid=7066]]], state=connected] 2024-04-23T21:44:48.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7067], [object_number=6, tag_tid=1, entry_tid=7066]]], state=connected] 2024-04-23T21:44:48.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:48.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:44:48.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_23 2024-04-23T21:44:48.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:48.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:48.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:44:48.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_23 '..*' 2024-04-23T21:44:48.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:48.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:48.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:44:48.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:48.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_23.status 2024-04-23T21:44:48.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_23.status 2024-04-23T21:44:48.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:48.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:48.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_23 2024-04-23T21:44:48.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:48.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:48.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:48.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:48.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7067], [object_number=6, tag_tid=1, entry_tid=7066]]], state=connected] 2024-04-23T21:44:48.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7067], [object_number=6, tag_tid=1, entry_tid=7066]]], state=connected] 2024-04-23T21:44:48.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:48.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:48.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:48.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:48.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:48.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:44:49.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_23 2024-04-23T21:44:49.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:49.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:49.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:44:49.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:49.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_23 ']' 2024-04-23T21:44:49.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_23' 2024-04-23T21:44:49.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_23 2024-04-23T21:44:49.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:49.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:49.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:49.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:49.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:49.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:49.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:49.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:49.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:49.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_23 2024-04-23T21:44:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:49.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:49.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:49.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:49.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:49.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:49.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:49.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:49.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:49.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:49.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:49.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:49.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:49.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_23 2024-04-23T21:44:49.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:44:49.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:49.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:49.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:49.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:44:49.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:49.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:49.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:49.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:44:49.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:49.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:49.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:49.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:44:49.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:49.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:49.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:44:49.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:49.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:49.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:49.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:49.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:49.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:49.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:49.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:49.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:49.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:49.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:49.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:49.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:49.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:49.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:49.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:49.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:49.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:49.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:49.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:49.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:49.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:49.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_23 2024-04-23T21:44:49.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:44:49.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:49.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:49.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:44:49.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:44:49.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:49.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:49.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:49.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:44:49.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:49.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:49.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:49.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:44:49.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:49.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:44:49.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:44:49.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:49.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:49.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:44:49.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:44:49.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:49.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:49.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:49.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:49.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:49.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:49.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:49.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:44:49.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:44:49.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:44:49.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:49.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:49.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:49.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:49.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:44:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_23 2024-04-23T21:44:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:44:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_23 '' 2024-04-23T21:44:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:44:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:44:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_23.status 2024-04-23T21:44:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_23.status 2024-04-23T21:44:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:49.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:49.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_23 2024-04-23T21:44:49.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7067], [object_number=6, tag_tid=1, entry_tid=7066]]], state=connected] 2024-04-23T21:44:49.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7067], [object_number=6, tag_tid=1, entry_tid=7066]]], state=connected] 2024-04-23T21:44:49.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:49.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:44:49.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_23 2024-04-23T21:44:49.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:49.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:49.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:44:49.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_23 '..*' 2024-04-23T21:44:49.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:44:49.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:44:49.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:44:49.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:44:49.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_23.status 2024-04-23T21:44:49.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_23.status 2024-04-23T21:44:49.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:44:49.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:49.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_23 2024-04-23T21:44:49.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:49.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:44:49.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:44:49.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:44:49.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7067], [object_number=6, tag_tid=1, entry_tid=7066]]], state=connected] 2024-04-23T21:44:49.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7067], [object_number=6, tag_tid=1, entry_tid=7066]]], state=connected] 2024-04-23T21:44:49.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:44:49.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:49.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:49.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:44:49.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:44:49.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:44:50.288 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9221859696752134) > weight (0.25) 2024-04-23T21:44:50.288 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.39481475630798013) > weight (0.25) 2024-04-23T21:44:50.288 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2024-04-23T21:44:50.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2024-04-23T21:44:50.289 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2024-04-23T21:44:50.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:44:50.290+0000 7f382f5c4640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 56750) UID: 1000 2024-04-23T21:44:50.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_23 2024-04-23T21:44:50.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:50.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:44:50.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:44:50.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:44:50.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_23 ']' 2024-04-23T21:44:50.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_23' 2024-04-23T21:44:50.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_23 2024-04-23T21:44:50.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:44:50.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:50.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:44:50.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:44:50.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:44:50.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:50.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:44:50.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:50.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:50.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:50.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_23 2024-04-23T21:44:50.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:50.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:50.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:50.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:50.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:50.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:50.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:50.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:50.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:50.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:50.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:50.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:50.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:50.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:50.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:50.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:50.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:50.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:50.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:50.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:50.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:50.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:50.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:50.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:50.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:50.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:50.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:50.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:50.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:50.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:50.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:50.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:50.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:50.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:50.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:50.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:50.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:50.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:50.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_23 2024-04-23T21:44:50.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:44:50.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:50.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:50.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:50.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:44:50.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:50.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:50.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:50.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:44:50.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:50.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:50.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:50.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:44:50.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:50.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:50.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:44:50.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:50.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:50.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:50.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:50.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:50.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:50.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:50.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:50.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:50.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:50.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:50.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:50.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:50.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:50.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:50.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:50.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:50.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:50.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:50.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:50.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:50.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:50.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_23 2024-04-23T21:44:50.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:44:50.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:50.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:50.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:44:50.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:44:50.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:50.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:50.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:50.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:44:50.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:50.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:50.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:50.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:44:50.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:50.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:44:50.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:44:50.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:50.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:50.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:44:50.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:44:50.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:50.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:50.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:50.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:50.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:50.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:50.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:50.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:44:50.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:44:50.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:44:50.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:50.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:50.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:50.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:50.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2024-04-23T21:44:50.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:50.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_23 2024-04-23T21:44:50.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:44:50.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:50.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:50.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:44:50.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:44:50.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:50.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:50.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:50.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:44:50.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:50.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:50.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:50.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:44:50.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:50.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:44:50.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:44:50.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:50.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:50.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:44:50.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:44:50.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:50.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:50.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:50.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:50.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:50.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:50.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:50.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:44:50.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:44:50.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:44:50.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:50.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:50.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:50.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:50.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:50.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:50.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:50.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:50.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:44:51.289 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2024-04-23T21:44:51.289 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:44:51.290 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2024-04-23T21:44:51.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2024-04-23T21:44:51.290 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2024-04-23T21:44:51.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2024-04-23T21:44:51.293 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2024-04-23T21:44:51.528 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:44:51.528+0000 7f9212057640 -1 reset not still connected to 0x5614e60ed6c0 2024-04-23T21:44:51.529 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:44:51.528+0000 7f9212057640 -1 reset not still connected to 0x5614f6485520 2024-04-23T21:44:51.529 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:44:51.528+0000 7f9212057640 -1 reset not still connected to 0x56150a2df110 2024-04-23T21:44:51.529 INFO:tasks.ceph.cluster2.osd.2.smithi150.stderr:2024-04-23T21:44:51.529+0000 7fe97bfc2640 -1 reset not still connected to 0x55a5ddd296c0 2024-04-23T21:44:51.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:44:51.737+0000 7f438fb63640 -1 rbd::mirror::image_replayer::journal::Replayer: 0x55973e524e00 get_replay_status: replay not running 2024-04-23T21:44:51.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:44:51.738+0000 7f438fb63640 -1 rbd::mirror::image_replayer::journal::Replayer: 0x55973eff6380 get_replay_status: replay not running 2024-04-23T21:44:51.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:44:51.769+0000 7f5e6175b640 -1 rbd::mirror::image_replayer::journal::Replayer: 0x55fdf7498e00 get_replay_status: replay not running 2024-04-23T21:44:51.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:51.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:51.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:51.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_23 2024-04-23T21:44:51.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:51.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:51.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:51.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:51.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:51.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:51.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:51.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:51.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:51.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:51.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:51.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:51.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:51.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:51.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:51.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:51.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:51.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:51.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:51.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:51.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:51.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:51.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:51.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:51.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:51.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:51.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:51.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:51.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:51.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:52.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:52.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:52.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:52.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:52.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:52.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:52.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_23 2024-04-23T21:44:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:44:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:44:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:44:52.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:52.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:52.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:52.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:44:52.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:52.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:52.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:44:52.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:52.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:52.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:52.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:52.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:52.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:52.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:52.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:52.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:52.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:52.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:52.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:52.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:52.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:52.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:52.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:52.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:52.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:52.269 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:44:52.269+0000 7f83e7350640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 1 2024-04-23T21:44:52.275 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:44:52.274+0000 7f83ea356640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 2 2024-04-23T21:44:52.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:52.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:52.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:52.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:52.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_23 2024-04-23T21:44:52.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:44:52.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:52.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:52.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:44:52.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:44:52.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:52.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:52.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:52.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:44:52.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:52.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:52.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:52.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:44:52.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:52.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:44:52.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:44:52.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:52.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:52.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:44:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:44:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:44:52.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:44:52.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:44:52.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:52.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:52.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:52.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:52.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:52.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:52.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:52.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:52.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_23 2024-04-23T21:44:52.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:44:52.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:52.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:52.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:44:52.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:44:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:44:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:44:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:44:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:44:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:44:52.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:44:52.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:52.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:52.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:52.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:52.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:52.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:52.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:52.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:44:52.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:44:52.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:44:52.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:52.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:52.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:52.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:52.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:52.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:52.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:52.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:52.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:44:54.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:54.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_23 2024-04-23T21:44:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:54.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:54.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:54.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:54.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:54.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:54.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:54.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:54.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:54.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:54.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:54.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:54.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:54.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:54.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_23 2024-04-23T21:44:54.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:44:54.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:44:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:44:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:44:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:44:54.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:54.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:54.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:54.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:54.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:54.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:54.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:54.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:54.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:54.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:54.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:54.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:54.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:54.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:54.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:54.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:54.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:54.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:54.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:54.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:54.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:54.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:54.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_23 2024-04-23T21:44:54.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:44:54.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:54.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:54.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:44:54.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:44:54.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:44:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:44:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:44:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:44:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:44:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:44:54.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:54.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:54.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:54.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:54.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:54.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:54.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:54.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:44:54.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:44:54.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:44:54.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:54.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:54.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:54.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:55.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:55.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:55.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_23 2024-04-23T21:44:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:44:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:44:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:44:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:44:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:44:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:44:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:44:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:44:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:44:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:55.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:44:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:44:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:44:55.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:55.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:55.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:55.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:55.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:55.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:55.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:55.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:55.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 4 2024-04-23T21:44:57.894 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2024-04-23T21:44:57.894 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2024-04-23T21:44:57.894 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2024-04-23T21:44:57.895 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:44:57.896+0000 7ff9f1102640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 63030) UID: 1000 2024-04-23T21:44:59.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:44:59.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:44:59.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:59.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_23 2024-04-23T21:44:59.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:44:59.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:59.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:59.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:44:59.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:44:59.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:59.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:59.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:59.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:44:59.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:59.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:59.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:59.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:44:59.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:59.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:44:59.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:44:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:44:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:44:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:44:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:44:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:44:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:44:59.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:44:59.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:59.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:59.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:44:59.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:59.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:59.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:59.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:59.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:59.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_23 2024-04-23T21:44:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:44:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:44:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:44:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:44:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:44:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:44:59.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:44:59.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:59.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:59.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:44:59.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:44:59.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:59.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:59.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:59.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:44:59.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:44:59.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:59.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:59.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:44:59.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:44:59.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:44:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:44:59.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:59.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:59.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:59.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:59.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:59.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_23 2024-04-23T21:44:59.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:44:59.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:59.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:59.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:44:59.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:44:59.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:59.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:59.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:59.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:44:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:44:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:44:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:44:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:44:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:44:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:59.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:44:59.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:44:59.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:59.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:59.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:44:59.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:44:59.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:44:59.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:59.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:59.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:44:59.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:59.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:59.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:44:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_23 2024-04-23T21:44:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:44:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:44:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:44:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:44:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:44:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:44:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:44:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:44:59.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:44:59.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:59.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:44:59.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:59.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:44:59.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:59.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:44:59.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:44:59.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:44:59.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:44:59.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:44:59.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:44:59.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:44:59.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:44:59.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:44:59.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:44:59.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:44:59.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:44:59.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:44:59.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:44:59.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:44:59.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:44:59.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:59.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:59.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:44:59.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_23 2024-04-23T21:44:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:44:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:44:59.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:44:59.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:44:59.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 8 2024-04-23T21:45:03.895 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2024-04-23T21:45:03.895 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:45:03.896 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2024-04-23T21:45:03.896 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2024-04-23T21:45:03.896 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2024-04-23T21:45:03.899 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2024-04-23T21:45:03.899 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2024-04-23T21:45:07.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:07.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:07.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:07.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_23 2024-04-23T21:45:07.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:07.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:07.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:07.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:07.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:07.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:07.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:07.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:07.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:07.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:07.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:07.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:07.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:07.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:07.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:07.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:07.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:07.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:07.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:07.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:07.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:07.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:07.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:07.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:07.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:07.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:07.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:07.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:07.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:07.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_23 2024-04-23T21:45:07.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:45:07.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:07.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:07.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:07.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_23 2024-04-23T21:45:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:45:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:45:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:45:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:45:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:07.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:45:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:07.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:07.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:07.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:07.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:07.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_23 2024-04-23T21:45:07.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:45:07.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:07.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:07.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_23 2024-04-23T21:45:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:45:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:45:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:45:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:45:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:45:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:07.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:45:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:45:07.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:45:07.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:45:07.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:45:07.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:45:07.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_23 2024-04-23T21:45:07.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:45:07.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:07.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_23 2024-04-23T21:45:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:45:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:45:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:45:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:45:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:45:07.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:07.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:07.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:45:07.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:45:07.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:45:07.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:45:07.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:45:07.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:45:07.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_23 2024-04-23T21:45:07.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:45:07.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:07.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:07.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:07.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 8 2024-04-23T21:45:15.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:15.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:15.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:15.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_23 2024-04-23T21:45:15.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:15.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:15.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:15.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:15.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:15.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:15.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:15.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:15.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:15.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:15.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:15.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:15.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:15.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:15.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:15.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:15.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:15.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:15.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_23 2024-04-23T21:45:16.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:45:16.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:16.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:16.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_23 2024-04-23T21:45:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:45:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:45:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:45:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:16.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:45:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:45:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:16.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:16.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:16.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:16.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:16.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:16.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_23 2024-04-23T21:45:16.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:45:16.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:16.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:16.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:16.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_23 2024-04-23T21:45:16.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:45:16.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:16.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:16.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:45:16.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:45:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:45:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:45:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:45:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:45:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:45:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:45:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:45:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:45:16.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:16.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:16.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:16.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:45:16.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:45:16.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:16.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:16.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:45:16.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:45:16.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:45:16.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:45:16.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:45:16.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:45:16.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_23 2024-04-23T21:45:16.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:45:16.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:16.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:16.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:16.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_23 2024-04-23T21:45:16.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:45:16.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:16.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:16.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:45:16.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:45:16.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:16.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:45:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:45:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:45:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:45:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:45:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:45:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:45:16.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:45:16.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:16.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:16.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:16.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:45:16.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:45:16.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:16.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:16.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:45:16.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:45:16.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:45:16.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:45:16.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:45:16.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:45:16.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_23 2024-04-23T21:45:16.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:45:16.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:16.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 8 2024-04-23T21:45:24.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:24.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:24.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:24.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_23 2024-04-23T21:45:24.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:24.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:24.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:24.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:24.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:24.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_23 2024-04-23T21:45:24.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2024-04-23T21:45:24.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:24.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:24.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:24.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_23 2024-04-23T21:45:24.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:45:24.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:24.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:24.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:24.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:45:24.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:24.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:24.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:24.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:45:24.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:24.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:24.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:24.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:45:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:45:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:24.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:24.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:24.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:24.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:24.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:24.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:24.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:24.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_23 2024-04-23T21:45:24.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:24.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_23 2024-04-23T21:45:24.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:24.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:24.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:45:24.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_23 '' 2024-04-23T21:45:24.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:24.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:24.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:45:24.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:24.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_23.status 2024-04-23T21:45:24.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_23.status 2024-04-23T21:45:24.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:24.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:45:24.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_23 2024-04-23T21:45:24.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:45:24.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:24.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:24.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:24.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7067], [object_number=6, tag_tid=1, entry_tid=7066]]], state=connected] 2024-04-23T21:45:24.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7067], [object_number=6, tag_tid=1, entry_tid=7066]]], state=connected] 2024-04-23T21:45:24.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:45:24.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:45:24.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_23 2024-04-23T21:45:24.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:24.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:24.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:45:24.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_23 '..*' 2024-04-23T21:45:24.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:24.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:24.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:45:24.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:24.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_23.status 2024-04-23T21:45:24.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_23.status 2024-04-23T21:45:24.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:24.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:45:24.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_23 2024-04-23T21:45:24.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:45:24.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:24.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:24.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:24.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7067], [object_number=6, tag_tid=1, entry_tid=7066]]], state=connected] 2024-04-23T21:45:24.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7067], [object_number=6, tag_tid=1, entry_tid=7066]]], state=connected] 2024-04-23T21:45:24.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.status 2024-04-23T21:45:24.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:45:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:45:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_23 snap 2024-04-23T21:45:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:45:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:45:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:45:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:45:24.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:45:25.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_23@snap 2024-04-23T21:45:25.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_23': 2024-04-23T21:45:25.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:45:25.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:45:25.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:45:25.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 11b850bf8e3e 2024-04-23T21:45:25.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.11b850bf8e3e 2024-04-23T21:45:25.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:45:25.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:45:25.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:45:25.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:45:25.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:35:33 2024 2024-04-23T21:45:25.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:35:33 2024 2024-04-23T21:45:25.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:37:50 2024 2024-04-23T21:45:25.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:45:25.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 11b850bf8e3e 2024-04-23T21:45:25.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:45:25.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:45:25.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: f8bc7596-4246-40de-af59-cdda7ef6bb72 2024-04-23T21:45:25.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:45:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_23 snap 2024-04-23T21:45:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:45:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:45:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:45:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.export 2024-04-23T21:45:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_23.export 2024-04-23T21:45:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_23.export 2024-04-23T21:45:25.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_23@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.export 2024-04-23T21:45:25.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:45:25.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_23@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_23.export 2024-04-23T21:45:26.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:45:26.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_23.export 2024-04-23T21:45:26.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_23.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_23.export 2024-04-23T21:45:26.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:26.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:45:26.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_24 2024-04-23T21:45:26.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_24 snap 2024-04-23T21:45:26.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:45:26.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:26.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:45:26.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:45:26.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_24@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:45:27.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_24 2024-04-23T21:45:27.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:27.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:27.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:45:27.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_24 started 2024-04-23T21:45:27.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:27.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:27.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:45:27.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:45:27.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:45:27.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:27.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:45:28.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_24 started 2024-04-23T21:45:28.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:28.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:28.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:45:28.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:45:28.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:45:28.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:45:28.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_24 2024-04-23T21:45:28.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:45:28.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:45:28.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:28.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:45:28.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:45:28.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:45:28.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:28.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:45:28.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:28.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:45:28.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:28.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_24 2024-04-23T21:45:28.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:45:28.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:45:28.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:28.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:28.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:28.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:45:28.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:28.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:28.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:28.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:28.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:45:28.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:45:28.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:28.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:28.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:28.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:45:28.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:45:28.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:45:28.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:28.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:45:28.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:45:28.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:45:28.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:45:28.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:28.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:28.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:45:28.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:45:28.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:45:28.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:45:28.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:28.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:28.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:28.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:28.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_24 2024-04-23T21:45:29.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_24'] 2024-04-23T21:45:29.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:45:29.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:29.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:29.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_24 2024-04-23T21:45:29.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:45:29.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:45:29.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:29.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:29.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:29.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:45:29.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:29.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:29.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:29.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:29.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:45:29.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:45:29.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:29.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:29.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:29.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:45:29.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:45:29.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:45:29.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:29.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:45:29.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:45:29.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:45:29.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:45:29.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:29.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:29.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:45:29.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:45:29.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:45:29.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:45:29.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:29.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:29.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:29.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:29.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_24 2024-04-23T21:45:29.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:45:29.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:45:29.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:45:29.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:45:29.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:45:29.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:45:29.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:45:29.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:29.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_24 2024-04-23T21:45:29.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:45:29.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:45:29.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:29.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:45:29.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:45:29.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_24 2024-04-23T21:45:29.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:45:29.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:45:29.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:29.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:45:29.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:45:29.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:45:29.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:45:29.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:29.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:45:29.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_24 2024-04-23T21:45:29.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:29.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:29.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:45:29.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:29.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_24 ']' 2024-04-23T21:45:29.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_24' 2024-04-23T21:45:29.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_24 2024-04-23T21:45:29.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:29.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:29.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:29.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:29.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:29.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:29.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:29.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:29.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:29.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:29.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_24 2024-04-23T21:45:29.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:29.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:29.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:29.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:29.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:29.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:29.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:29.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:29.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:29.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:29.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:29.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:29.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:29.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:29.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:29.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:29.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_24 2024-04-23T21:45:29.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_24'] 2024-04-23T21:45:29.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:29.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:29.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:29.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_24 2024-04-23T21:45:29.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:45:29.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:45:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:45:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:45:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:29.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:45:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:29.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:29.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:29.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:29.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:29.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_24 2024-04-23T21:45:29.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:29.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_24 2024-04-23T21:45:29.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:29.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:29.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:29.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_24 '' 2024-04-23T21:45:29.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:29.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:29.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:29.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:29.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_24.status 2024-04-23T21:45:29.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_24.status 2024-04-23T21:45:29.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:29.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:29.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_24 2024-04-23T21:45:29.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:29.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:29.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:29.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:29.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022]]], state=connected] 2024-04-23T21:45:29.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=5024], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022], [object_number=5, tag_tid=1, entry_tid=5021]]], state=connected] 2024-04-23T21:45:29.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:29.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:29.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_24 2024-04-23T21:45:29.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:29.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:29.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:29.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_24 '..*' 2024-04-23T21:45:29.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:29.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:29.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:29.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:29.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_24.status 2024-04-23T21:45:29.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_24.status 2024-04-23T21:45:29.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:29.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:29.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_24 2024-04-23T21:45:29.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:29.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:29.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:29.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:29.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022]]], state=connected] 2024-04-23T21:45:29.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=5024], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022], [object_number=5, tag_tid=1, entry_tid=5021]]], state=connected] 2024-04-23T21:45:29.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:29.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=4, tag_tid=1, entry_tid=5024' 2024-04-23T21:45:29.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=4, tag_tid=1, entry_tid=5024' 2024-04-23T21:45:29.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=4, tag_tid=1, entry_tid=5024' '!=' '' 2024-04-23T21:45:29.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:45:29.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=4, tag_tid=1, entry_tid=5024' = '' 2024-04-23T21:45:29.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=4, tag_tid=1, entry_tid=5024' 2024-04-23T21:45:29.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:29.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:45:29.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:45:29.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:45:29.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=4, tag_tid=1, entry_tid=5024' 2024-04-23T21:45:29.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:45:29.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:45:29.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:45:29.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:29.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:45:29.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:45:29.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:45:29.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=4, tag_tid=1, entry_tid=5024' 2024-04-23T21:45:29.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:45:29.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:45:29.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=5024 2024-04-23T21:45:29.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 5024 2024-04-23T21:45:29.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:45:29.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:29.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:45:29.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_24 2024-04-23T21:45:29.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:29.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:29.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:45:29.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:29.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_24 ']' 2024-04-23T21:45:29.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_24' 2024-04-23T21:45:29.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_24 2024-04-23T21:45:29.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:29.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:29.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:29.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:29.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:29.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:29.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:29.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:29.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_24 2024-04-23T21:45:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:29.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:29.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:29.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:29.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:29.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:29.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:29.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:29.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:29.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:29.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:29.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:29.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_24 2024-04-23T21:45:30.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_24'] 2024-04-23T21:45:30.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:30.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:30.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:30.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_24 2024-04-23T21:45:30.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:45:30.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:30.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:30.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:30.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:45:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:45:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:45:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:45:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:30.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:30.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:30.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:30.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:30.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:30.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_24 2024-04-23T21:45:30.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:30.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_24 2024-04-23T21:45:30.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:30.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:30.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:30.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_24 '' 2024-04-23T21:45:30.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:30.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:30.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:30.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:30.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_24.status 2024-04-23T21:45:30.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_24.status 2024-04-23T21:45:30.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:30.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:30.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_24 2024-04-23T21:45:30.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:30.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:30.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:30.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:30.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022]]], state=connected] 2024-04-23T21:45:30.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=5024], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022], [object_number=5, tag_tid=1, entry_tid=5021]]], state=connected] 2024-04-23T21:45:30.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:30.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:30.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_24 2024-04-23T21:45:30.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:30.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:30.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:30.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_24 '..*' 2024-04-23T21:45:30.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:30.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:30.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:30.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:30.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_24.status 2024-04-23T21:45:30.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_24.status 2024-04-23T21:45:30.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:30.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:30.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_24 2024-04-23T21:45:30.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:30.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:30.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:30.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:30.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022]]], state=connected] 2024-04-23T21:45:30.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=5024], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022], [object_number=5, tag_tid=1, entry_tid=5021]]], state=connected] 2024-04-23T21:45:30.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:30.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=4, tag_tid=1, entry_tid=5024' 2024-04-23T21:45:30.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=4, tag_tid=1, entry_tid=5024' 2024-04-23T21:45:30.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=4, tag_tid=1, entry_tid=5024' '!=' 'object_number=4, tag_tid=1, entry_tid=5024' 2024-04-23T21:45:30.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:30.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:45:30.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_24 2024-04-23T21:45:30.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:30.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:30.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:45:30.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:30.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_24 ']' 2024-04-23T21:45:30.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_24' 2024-04-23T21:45:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_24 2024-04-23T21:45:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:30.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:30.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:30.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_24 2024-04-23T21:45:30.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:30.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:30.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:30.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:30.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:30.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:30.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:30.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:30.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:30.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:30.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:30.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:30.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:30.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:30.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:30.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:30.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:30.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:30.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:30.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:30.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:30.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:30.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:30.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:30.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:30.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:30.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:30.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:30.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:30.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:30.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:30.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:30.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:30.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_24 2024-04-23T21:45:30.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_24'] 2024-04-23T21:45:30.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:30.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:30.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:30.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_24 2024-04-23T21:45:30.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:45:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:45:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:45:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:45:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:30.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:45:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:30.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:30.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:30.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:30.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:30.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:30.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_24 2024-04-23T21:45:30.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:30.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_24 2024-04-23T21:45:30.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:30.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:30.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:30.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_24 '' 2024-04-23T21:45:30.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:30.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:30.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:30.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:30.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_24.status 2024-04-23T21:45:30.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_24.status 2024-04-23T21:45:30.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:30.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:30.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_24 2024-04-23T21:45:30.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:30.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:30.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:30.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:30.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022]]], state=connected] 2024-04-23T21:45:30.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=5024], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022], [object_number=5, tag_tid=1, entry_tid=5021]]], state=connected] 2024-04-23T21:45:30.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:30.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:30.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_24 2024-04-23T21:45:30.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:30.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:30.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:30.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_24 '..*' 2024-04-23T21:45:30.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:30.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:30.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:30.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:30.906 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_24.status 2024-04-23T21:45:30.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_24.status 2024-04-23T21:45:30.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:30.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:30.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_24 2024-04-23T21:45:30.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:30.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:30.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:30.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:30.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022]]], state=connected] 2024-04-23T21:45:30.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=5024], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022], [object_number=5, tag_tid=1, entry_tid=5021]]], state=connected] 2024-04-23T21:45:30.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:30.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=4, tag_tid=1, entry_tid=5024' 2024-04-23T21:45:30.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=4, tag_tid=1, entry_tid=5024' 2024-04-23T21:45:30.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=4, tag_tid=1, entry_tid=5024' '!=' 'object_number=4, tag_tid=1, entry_tid=5024' 2024-04-23T21:45:30.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:30.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:45:31.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_24 2024-04-23T21:45:31.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:31.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:31.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:45:31.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:31.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_24 ']' 2024-04-23T21:45:31.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_24' 2024-04-23T21:45:31.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_24 2024-04-23T21:45:31.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:31.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:31.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:31.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:31.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:31.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:31.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:31.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:31.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:31.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:31.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_24 2024-04-23T21:45:31.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:31.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:31.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:31.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:31.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:31.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:31.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:31.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:31.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:31.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:31.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:31.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:31.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:31.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:31.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:31.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:31.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:31.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:31.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:31.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:31.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:31.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:31.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_24 2024-04-23T21:45:31.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_24'] 2024-04-23T21:45:31.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:31.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_24 2024-04-23T21:45:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:45:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:45:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:45:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:45:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:45:31.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:31.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:31.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:31.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:31.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:31.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_24 2024-04-23T21:45:31.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:31.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_24 2024-04-23T21:45:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_24 '' 2024-04-23T21:45:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_24.status 2024-04-23T21:45:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_24.status 2024-04-23T21:45:31.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:31.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:31.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_24 2024-04-23T21:45:31.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:32.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:32.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:32.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:32.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022]]], state=connected] 2024-04-23T21:45:32.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=5024], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022], [object_number=5, tag_tid=1, entry_tid=5021]]], state=connected] 2024-04-23T21:45:32.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:32.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_24 2024-04-23T21:45:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_24 '..*' 2024-04-23T21:45:32.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:32.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:32.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:32.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:32.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_24.status 2024-04-23T21:45:32.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_24.status 2024-04-23T21:45:32.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:32.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:32.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_24 2024-04-23T21:45:32.028 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:32.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:32.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:32.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:32.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022]]], state=connected] 2024-04-23T21:45:32.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=4, tag_tid=1, entry_tid=5024], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022], [object_number=5, tag_tid=1, entry_tid=5021]]], state=connected] 2024-04-23T21:45:32.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:32.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=4, tag_tid=1, entry_tid=5024' 2024-04-23T21:45:32.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=4, tag_tid=1, entry_tid=5024' 2024-04-23T21:45:32.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=4, tag_tid=1, entry_tid=5024' '!=' 'object_number=4, tag_tid=1, entry_tid=5024' 2024-04-23T21:45:32.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:32.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:45:33.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_24 2024-04-23T21:45:33.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:33.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:33.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:45:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_24 ']' 2024-04-23T21:45:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_24' 2024-04-23T21:45:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_24 2024-04-23T21:45:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:33.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:33.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:33.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_24 2024-04-23T21:45:33.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:33.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:33.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:33.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:33.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:33.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:33.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:33.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:33.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:33.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:33.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:33.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:33.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:33.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:33.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:33.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:33.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:33.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:33.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:33.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_24 2024-04-23T21:45:33.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_24'] 2024-04-23T21:45:33.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:33.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:33.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:33.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_24 2024-04-23T21:45:33.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:45:33.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:33.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:33.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:33.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:45:33.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:45:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:45:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:45:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:33.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:33.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:33.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:33.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:33.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:33.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:33.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:33.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:33.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:33.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:33.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:33.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:33.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_24 2024-04-23T21:45:33.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:33.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_24 2024-04-23T21:45:33.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:33.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:33.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:33.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_24 '' 2024-04-23T21:45:33.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:33.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:33.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:33.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:33.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_24.status 2024-04-23T21:45:33.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_24.status 2024-04-23T21:45:33.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:33.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:33.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_24 2024-04-23T21:45:33.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:33.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:33.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:33.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:33.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022]]], state=connected] 2024-04-23T21:45:33.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022]]], state=connected] 2024-04-23T21:45:33.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:33.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:33.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_24 2024-04-23T21:45:33.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:33.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:33.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:33.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_24 '..*' 2024-04-23T21:45:33.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:33.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:33.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:45:33.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:33.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_24.status 2024-04-23T21:45:33.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_24.status 2024-04-23T21:45:33.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:33.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:33.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_24 2024-04-23T21:45:33.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022]]], state=connected] 2024-04-23T21:45:33.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=5023], [object_number=6, tag_tid=1, entry_tid=5022]]], state=connected] 2024-04-23T21:45:33.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.status 2024-04-23T21:45:33.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:33.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:33.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:33.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_24 snap 2024-04-23T21:45:33.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:34.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:34.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:45:34.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:45:34.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:45:34.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:45:34.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:45:35.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_24@snap 2024-04-23T21:45:35.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_24': 2024-04-23T21:45:35.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:45:35.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:45:35.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:45:35.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 121d7be5b7e3 2024-04-23T21:45:35.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.121d7be5b7e3 2024-04-23T21:45:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:45:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:45:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:45:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:45:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:35:38 2024 2024-04-23T21:45:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:35:38 2024 2024-04-23T21:45:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:35:38 2024 2024-04-23T21:45:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:45:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 121d7be5b7e3 2024-04-23T21:45:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:45:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:45:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: c2875430-58b7-41e7-98df-550108816e81 2024-04-23T21:45:35.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:45:35.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:35.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_24 snap 2024-04-23T21:45:35.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:35.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:45:35.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:45:35.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:45:35.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.export 2024-04-23T21:45:35.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_24.export 2024-04-23T21:45:35.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_24.export 2024-04-23T21:45:35.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_24@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.export 2024-04-23T21:45:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:45:35.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_24@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_24.export 2024-04-23T21:45:35.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:45:35.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_24.export 2024-04-23T21:45:35.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_24.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_24.export 2024-04-23T21:45:35.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:35.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:45:35.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_25 2024-04-23T21:45:35.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_25 snap 2024-04-23T21:45:35.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:45:35.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:35.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:45:35.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:45:35.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_25@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:45:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_25 2024-04-23T21:45:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:38.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:45:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_25 started 2024-04-23T21:45:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:45:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:45:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:45:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:45:39.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_25 started 2024-04-23T21:45:39.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:39.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:39.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:45:39.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:45:39.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:45:39.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:45:39.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_25 2024-04-23T21:45:39.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:45:39.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:45:39.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:45:39.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:39.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:45:39.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:45:39.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:39.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:45:39.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:39.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:45:39.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:39.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_25 2024-04-23T21:45:39.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:45:39.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:45:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:45:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:45:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:45:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:45:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:45:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:45:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:45:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:45:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:45:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:45:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:45:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:45:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:45:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:45:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:39.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:39.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:39.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:39.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_25 2024-04-23T21:45:39.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_25'] 2024-04-23T21:45:39.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:45:39.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:39.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:39.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_25 2024-04-23T21:45:39.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:45:39.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:45:39.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:39.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:39.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:45:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:45:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:45:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:45:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:45:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:45:39.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:45:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:45:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:45:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:45:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:45:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:45:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:45:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:45:39.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:39.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:39.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:39.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:39.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_25 2024-04-23T21:45:39.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:45:39.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:45:39.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:45:39.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:45:39.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:45:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:45:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:45:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_25 2024-04-23T21:45:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:45:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:45:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:45:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:45:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_25 2024-04-23T21:45:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:45:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:45:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:45:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:45:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:45:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:45:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:45:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_25 2024-04-23T21:45:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:45:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_25 ']' 2024-04-23T21:45:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_25' 2024-04-23T21:45:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_25 2024-04-23T21:45:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_25 2024-04-23T21:45:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:39.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:39.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:39.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:39.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:39.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:39.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:39.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:39.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:39.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:39.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:39.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:39.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:39.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_25 2024-04-23T21:45:39.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_25'] 2024-04-23T21:45:39.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:39.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_25 2024-04-23T21:45:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:45:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:39.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:45:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:45:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:45:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:45:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:39.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:39.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:39.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:39.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:39.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:39.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:39.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:39.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:39.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:39.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:39.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:39.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:39.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:39.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:39.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:39.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:39.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_25 2024-04-23T21:45:40.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:40.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_25 2024-04-23T21:45:40.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:40.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:40.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:40.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_25 '' 2024-04-23T21:45:40.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:40.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:40.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:40.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:40.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_25.status 2024-04-23T21:45:40.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_25.status 2024-04-23T21:45:40.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:40.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:40.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_25 2024-04-23T21:45:40.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:40.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:45:40.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:45:40.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:40.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=12414], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:40.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=12414], [object_number=9, tag_tid=1, entry_tid=12413], [object_number=8, tag_tid=1, entry_tid=12412], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:40.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:40.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:45:40.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_25 2024-04-23T21:45:40.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:40.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:40.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:40.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_25 '..*' 2024-04-23T21:45:40.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:40.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:40.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:40.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:40.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_25.status 2024-04-23T21:45:40.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_25.status 2024-04-23T21:45:40.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:40.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:40.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_25 2024-04-23T21:45:40.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:40.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:45:40.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:45:40.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:40.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=12414], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:40.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=12414], [object_number=9, tag_tid=1, entry_tid=12413], [object_number=8, tag_tid=1, entry_tid=12412], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:40.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:40.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:40.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:40.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=12414' '!=' '' 2024-04-23T21:45:40.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:45:40.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=12414' = '' 2024-04-23T21:45:40.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:40.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:45:40.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:45:40.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:45:40.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:45:40.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:40.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:45:40.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:45:40.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:45:40.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:45:40.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:45:40.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:45:40.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:45:40.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:40.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:45:40.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:45:40.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=12414 2024-04-23T21:45:40.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 12414 2024-04-23T21:45:40.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:45:40.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:40.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:45:40.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_25 2024-04-23T21:45:40.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:40.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:40.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:45:40.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:40.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_25 ']' 2024-04-23T21:45:40.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_25' 2024-04-23T21:45:40.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_25 2024-04-23T21:45:40.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:40.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:40.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:40.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:40.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:40.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:40.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:40.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:40.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:40.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:40.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_25 2024-04-23T21:45:40.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:40.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:40.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:40.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:40.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:40.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:40.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:40.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:40.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:40.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:40.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:40.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:40.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:40.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:40.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:40.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:40.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:40.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:40.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:40.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:40.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:40.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_25 2024-04-23T21:45:40.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_25'] 2024-04-23T21:45:40.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:40.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:40.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:40.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_25 2024-04-23T21:45:40.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:45:40.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:40.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:40.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:40.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:45:40.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:40.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:40.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:40.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:45:40.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:40.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:40.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:40.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:45:40.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:40.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:40.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:45:40.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:40.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:40.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:40.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:40.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:40.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:40.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:40.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:40.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:40.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:40.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:40.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:40.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:40.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:40.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:40.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:40.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:40.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_25 2024-04-23T21:45:40.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:40.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_25 2024-04-23T21:45:40.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:40.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:40.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:40.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_25 '' 2024-04-23T21:45:40.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:40.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:40.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:40.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:40.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_25.status 2024-04-23T21:45:40.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_25.status 2024-04-23T21:45:40.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:40.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:40.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_25 2024-04-23T21:45:40.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:40.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:45:40.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:45:40.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:40.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=12414], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:40.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=12414], [object_number=9, tag_tid=1, entry_tid=12413], [object_number=8, tag_tid=1, entry_tid=12412], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:40.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:40.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:45:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_25 2024-04-23T21:45:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_25 '..*' 2024-04-23T21:45:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:40.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_25.status 2024-04-23T21:45:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_25.status 2024-04-23T21:45:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_25 2024-04-23T21:45:40.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:45:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:45:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=12414], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:40.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=12414], [object_number=9, tag_tid=1, entry_tid=12413], [object_number=8, tag_tid=1, entry_tid=12412], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:40.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:40.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:40.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:40.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=12414' '!=' 'object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:40.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:40.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:45:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_25 2024-04-23T21:45:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:45:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_25 ']' 2024-04-23T21:45:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_25' 2024-04-23T21:45:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_25 2024-04-23T21:45:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:41.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_25 2024-04-23T21:45:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:41.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:41.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:41.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:41.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_25 2024-04-23T21:45:41.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_25'] 2024-04-23T21:45:41.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:41.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:41.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_25 2024-04-23T21:45:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:45:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:45:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:45:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:41.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:45:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:45:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:41.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:41.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:41.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:41.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_25 2024-04-23T21:45:41.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_25 2024-04-23T21:45:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_25 '' 2024-04-23T21:45:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_25.status 2024-04-23T21:45:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_25.status 2024-04-23T21:45:41.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_25 2024-04-23T21:45:41.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:41.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:45:41.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:45:41.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:41.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=12414], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:41.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=12414], [object_number=9, tag_tid=1, entry_tid=12413], [object_number=8, tag_tid=1, entry_tid=12412], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:41.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:45:41.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_25 2024-04-23T21:45:41.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:41.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:41.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:41.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_25 '..*' 2024-04-23T21:45:41.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:41.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:41.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:41.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:41.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_25.status 2024-04-23T21:45:41.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_25.status 2024-04-23T21:45:41.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:41.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:41.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_25 2024-04-23T21:45:41.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:41.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:45:41.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:45:41.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:41.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=12414], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:41.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=12414], [object_number=9, tag_tid=1, entry_tid=12413], [object_number=8, tag_tid=1, entry_tid=12412], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:41.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:41.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:41.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:41.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=12414' '!=' 'object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:41.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:41.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:45:42.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_25 2024-04-23T21:45:42.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:42.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:42.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:45:42.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:42.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_25 ']' 2024-04-23T21:45:42.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_25' 2024-04-23T21:45:42.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_25 2024-04-23T21:45:42.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:42.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:42.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:42.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:42.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_25 2024-04-23T21:45:42.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:42.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:42.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:42.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:42.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:42.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:42.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:42.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:42.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:42.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:42.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:42.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:42.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:42.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:42.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:42.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:42.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:42.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:42.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:42.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:42.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:42.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:42.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:42.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:42.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:42.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:42.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:42.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:42.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:42.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:42.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:42.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:42.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:42.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_25 2024-04-23T21:45:42.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_25'] 2024-04-23T21:45:42.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:42.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:42.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:42.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_25 2024-04-23T21:45:42.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:45:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:45:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:45:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:45:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:42.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:45:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:42.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:42.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:42.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:42.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:42.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_25 2024-04-23T21:45:42.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:42.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_25 2024-04-23T21:45:42.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:42.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:42.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:42.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_25 '' 2024-04-23T21:45:42.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:42.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:42.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:42.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:42.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_25.status 2024-04-23T21:45:42.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_25.status 2024-04-23T21:45:42.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:42.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:42.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_25 2024-04-23T21:45:42.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:42.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:45:42.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:45:42.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:42.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=12414], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:42.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=12414], [object_number=9, tag_tid=1, entry_tid=12413], [object_number=8, tag_tid=1, entry_tid=12412], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:42.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:42.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:45:42.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_25 2024-04-23T21:45:42.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:42.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:42.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:42.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_25 '..*' 2024-04-23T21:45:42.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:42.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:42.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:42.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:42.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_25.status 2024-04-23T21:45:42.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_25.status 2024-04-23T21:45:42.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:42.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:42.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_25 2024-04-23T21:45:42.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:45:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:45:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=12414], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=12414], [object_number=9, tag_tid=1, entry_tid=12413], [object_number=8, tag_tid=1, entry_tid=12412], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:42.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:42.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:42.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:42.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=12414' '!=' 'object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:42.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:42.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:45:44.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_25 2024-04-23T21:45:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:45:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_25 ']' 2024-04-23T21:45:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_25' 2024-04-23T21:45:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_25 2024-04-23T21:45:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:44.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:44.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:44.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:44.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_25 2024-04-23T21:45:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:44.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:44.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:44.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:44.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:44.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:44.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:44.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:44.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:44.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:44.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_25 2024-04-23T21:45:44.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_25'] 2024-04-23T21:45:44.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:44.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:44.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:44.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_25 2024-04-23T21:45:44.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:45:44.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:44.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:44.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:44.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:45:44.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:44.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:44.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:44.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:45:44.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:44.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:44.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:44.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:45:44.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:44.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:44.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:45:44.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:44.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:44.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:44.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:44.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:44.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:44.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:44.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:44.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:44.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:44.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:44.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:44.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:44.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:44.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:44.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:44.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:44.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_25 2024-04-23T21:45:44.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:44.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_25 2024-04-23T21:45:44.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:44.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:44.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:44.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_25 '' 2024-04-23T21:45:44.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:44.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:44.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:44.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:44.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_25.status 2024-04-23T21:45:44.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_25.status 2024-04-23T21:45:44.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:44.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:44.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_25 2024-04-23T21:45:44.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:44.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:45:44.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:45:44.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:44.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=12414], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:44.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=12414], [object_number=9, tag_tid=1, entry_tid=12413], [object_number=8, tag_tid=1, entry_tid=12412], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:44.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:44.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:45:44.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_25 2024-04-23T21:45:44.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:44.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:44.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:44.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_25 '..*' 2024-04-23T21:45:44.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:44.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:44.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:44.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:44.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_25.status 2024-04-23T21:45:44.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_25.status 2024-04-23T21:45:44.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:44.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:44.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_25 2024-04-23T21:45:44.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:44.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:45:44.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:45:44.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:44.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=12414], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:44.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=12414], [object_number=9, tag_tid=1, entry_tid=12413], [object_number=8, tag_tid=1, entry_tid=12412], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:44.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:44.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:44.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:44.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=12414' '!=' 'object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:44.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:44.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:45:46.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_25 2024-04-23T21:45:46.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:46.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:46.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:45:46.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:46.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_25 ']' 2024-04-23T21:45:46.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_25' 2024-04-23T21:45:46.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_25 2024-04-23T21:45:46.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:46.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:46.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:46.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:46.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:46.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:46.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:46.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:46.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:46.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:46.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_25 2024-04-23T21:45:46.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:46.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:46.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:46.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:46.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:46.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:46.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:46.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_25 2024-04-23T21:45:46.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_25'] 2024-04-23T21:45:46.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:46.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:46.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:46.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_25 2024-04-23T21:45:46.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:45:46.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:46.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:46.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:46.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:45:46.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:45:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:45:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:45:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:46.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:46.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:46.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:46.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:46.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:46.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:46.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:46.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:46.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:46.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:46.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:46.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:46.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:46.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_25 2024-04-23T21:45:46.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:46.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_25 2024-04-23T21:45:46.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:46.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:46.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:46.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_25 '' 2024-04-23T21:45:46.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:46.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:46.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:46.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:46.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_25.status 2024-04-23T21:45:46.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_25.status 2024-04-23T21:45:46.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:46.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:46.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_25 2024-04-23T21:45:46.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:46.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:45:46.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:45:46.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:46.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=12414], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:46.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=12414], [object_number=9, tag_tid=1, entry_tid=12413], [object_number=8, tag_tid=1, entry_tid=12412], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:46.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:46.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:45:46.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_25 2024-04-23T21:45:46.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:46.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:46.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:46.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_25 '..*' 2024-04-23T21:45:46.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:46.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:46.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:46.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:46.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_25.status 2024-04-23T21:45:46.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_25.status 2024-04-23T21:45:46.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:46.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:46.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_25 2024-04-23T21:45:46.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:45:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:45:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=12414], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=12414], [object_number=9, tag_tid=1, entry_tid=12413], [object_number=8, tag_tid=1, entry_tid=12412], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:46.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:46.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:46.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:46.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=12414' '!=' 'object_number=10, tag_tid=1, entry_tid=12414' 2024-04-23T21:45:46.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:46.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:45:48.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_25 2024-04-23T21:45:48.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:48.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:48.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:45:48.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:48.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_25 ']' 2024-04-23T21:45:48.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_25' 2024-04-23T21:45:48.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_25 2024-04-23T21:45:48.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:48.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:48.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:48.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:48.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:48.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:48.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:48.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:48.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:48.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:48.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_25 2024-04-23T21:45:48.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:48.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:48.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:48.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:48.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:48.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:48.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:48.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:48.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:48.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:48.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:48.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:48.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:48.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:48.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:48.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:48.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:48.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:48.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:48.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_25 2024-04-23T21:45:48.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_25'] 2024-04-23T21:45:48.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:45:48.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:45:48.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:48.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_25 2024-04-23T21:45:48.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:45:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:45:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:45:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:45:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:45:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:45:48.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:45:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:45:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:45:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:45:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:45:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:45:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:45:48.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:45:48.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:48.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:48.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:45:48.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_25 2024-04-23T21:45:48.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:48.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_25 2024-04-23T21:45:48.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:48.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:48.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:48.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_25 '' 2024-04-23T21:45:48.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:48.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:48.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:48.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:48.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_25.status 2024-04-23T21:45:48.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_25.status 2024-04-23T21:45:48.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:48.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:48.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_25 2024-04-23T21:45:48.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:48.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:45:48.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:45:48.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:48.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=12414], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:48.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=12414], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:48.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:48.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:45:48.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_25 2024-04-23T21:45:48.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:48.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:48.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:48.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_25 '..*' 2024-04-23T21:45:48.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:48.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:48.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:45:48.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:48.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_25.status 2024-04-23T21:45:48.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_25.status 2024-04-23T21:45:48.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:48.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:48.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_25 2024-04-23T21:45:48.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:49.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:45:49.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:45:49.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:49.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=12414], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:49.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=12414], [object_number=11, tag_tid=1, entry_tid=12411]]], state=connected] 2024-04-23T21:45:49.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.status 2024-04-23T21:45:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:45:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:45:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_25 snap 2024-04-23T21:45:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:45:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:45:49.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:45:49.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:45:49.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:45:50.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_25@snap 2024-04-23T21:45:50.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_25': 2024-04-23T21:45:50.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:45:50.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:45:50.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:45:50.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 121d415058f9 2024-04-23T21:45:50.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.121d415058f9 2024-04-23T21:45:50.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:45:50.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:45:50.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:45:50.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:45:50.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:35:46 2024 2024-04-23T21:45:50.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:35:46 2024 2024-04-23T21:45:50.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:36:47 2024 2024-04-23T21:45:50.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:45:50.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 121d415058f9 2024-04-23T21:45:50.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:45:50.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:45:50.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: cf46fb18-55b4-4a44-bd1a-963fbc7abcd0 2024-04-23T21:45:50.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:45:50.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:50.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_25 snap 2024-04-23T21:45:50.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:50.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:45:50.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:45:50.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:45:50.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.export 2024-04-23T21:45:50.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_25.export 2024-04-23T21:45:50.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_25.export 2024-04-23T21:45:50.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_25@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.export 2024-04-23T21:45:50.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:45:50.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_25@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_25.export 2024-04-23T21:45:50.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:45:50.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_25.export 2024-04-23T21:45:50.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_25.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_25.export 2024-04-23T21:45:50.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:50.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:45:50.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_26 2024-04-23T21:45:50.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_26 snap 2024-04-23T21:45:50.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:45:50.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:50.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:45:50.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:45:50.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_26@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:45:52.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_26 2024-04-23T21:45:52.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:52.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:52.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:45:52.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_26 started 2024-04-23T21:45:52.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:52.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:52.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:45:52.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:45:52.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:45:52.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:52.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:45:53.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_26 started 2024-04-23T21:45:53.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:53.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:53.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:45:53.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:45:53.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:45:53.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:45:53.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_26 2024-04-23T21:45:53.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:45:53.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:45:53.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:45:53.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:53.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:45:53.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:45:53.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:53.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:45:53.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:53.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:45:53.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:53.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_26 2024-04-23T21:45:53.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:45:53.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:45:53.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:53.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:53.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:53.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:45:53.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:53.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:53.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:53.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:53.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:45:53.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:45:53.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:53.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:53.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:53.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:45:53.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:45:53.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:45:53.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:53.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:45:53.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:45:53.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:45:53.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:45:53.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:53.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:53.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:45:53.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:45:53.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:45:53.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:45:53.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:53.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:53.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:53.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:53.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_26 2024-04-23T21:45:53.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:45:53.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:45:53.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:45:53.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:45:53.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:45:53.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:45:53.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:45:53.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:53.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_26 2024-04-23T21:45:53.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:45:53.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:45:53.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:53.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:45:53.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:45:53.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_26 2024-04-23T21:45:53.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:45:53.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:45:53.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:53.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:45:53.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:45:53.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:45:53.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:45:53.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:53.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:45:53.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_26 2024-04-23T21:45:53.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:53.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:53.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:45:53.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:53.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_26 ']' 2024-04-23T21:45:53.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_26' 2024-04-23T21:45:53.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_26 2024-04-23T21:45:53.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:53.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:53.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:53.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:53.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:53.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:53.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:53.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:53.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:53.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:53.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_26 2024-04-23T21:45:53.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:53.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:53.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:53.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:53.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:53.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:53.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:53.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:53.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:53.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:53.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:53.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:53.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:53.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:53.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:53.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:53.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:53.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:53.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:53.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:53.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:53.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:53.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:53.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:53.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:53.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:53.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:53.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:53.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:53.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:53.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:53.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:53.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:53.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_26 2024-04-23T21:45:53.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:53.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_26 2024-04-23T21:45:53.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:53.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:53.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:53.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_26 '' 2024-04-23T21:45:53.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:53.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:53.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:53.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:53.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_26.status 2024-04-23T21:45:53.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_26.status 2024-04-23T21:45:53.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:53.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:53.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_26 2024-04-23T21:45:53.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:53.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:53.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:53.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:53.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=7026], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:53.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=7026], [object_number=5, tag_tid=1, entry_tid=7025], [object_number=4, tag_tid=1, entry_tid=7024], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:53.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_26 2024-04-23T21:45:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_26 '..*' 2024-04-23T21:45:53.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_26.status 2024-04-23T21:45:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_26.status 2024-04-23T21:45:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:53.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:53.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_26 2024-04-23T21:45:53.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:53.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:53.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:53.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:53.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=7026], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:53.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=7026], [object_number=5, tag_tid=1, entry_tid=7025], [object_number=4, tag_tid=1, entry_tid=7024], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:53.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:53.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:53.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:53.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=7026' '!=' '' 2024-04-23T21:45:53.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:45:53.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=7026' = '' 2024-04-23T21:45:53.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:53.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:53.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:45:53.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:45:53.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:45:53.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:53.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:45:53.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:45:53.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:45:53.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:53.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:45:53.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:45:53.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:45:53.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:53.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:45:53.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:45:53.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=7026 2024-04-23T21:45:53.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 7026 2024-04-23T21:45:53.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:45:53.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:53.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:45:54.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_26 2024-04-23T21:45:54.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:54.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:54.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:45:54.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_26 ']' 2024-04-23T21:45:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_26' 2024-04-23T21:45:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_26 2024-04-23T21:45:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:54.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_26 2024-04-23T21:45:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:54.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:54.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:54.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:54.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:54.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:54.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:54.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:54.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:54.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:54.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:54.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:54.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:54.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:54.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:54.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:54.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:54.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:54.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_26 2024-04-23T21:45:54.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:54.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_26 2024-04-23T21:45:54.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:54.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:54.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:54.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_26 '' 2024-04-23T21:45:54.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:54.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:54.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:54.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:54.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_26.status 2024-04-23T21:45:54.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_26.status 2024-04-23T21:45:54.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:54.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:54.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_26 2024-04-23T21:45:54.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:54.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:54.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:54.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:54.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=7026], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:54.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=7026], [object_number=5, tag_tid=1, entry_tid=7025], [object_number=4, tag_tid=1, entry_tid=7024], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:54.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_26 2024-04-23T21:45:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_26 '..*' 2024-04-23T21:45:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:54.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_26.status 2024-04-23T21:45:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_26.status 2024-04-23T21:45:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_26 2024-04-23T21:45:54.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:54.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:54.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:54.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:54.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=7026], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:54.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=7026], [object_number=5, tag_tid=1, entry_tid=7025], [object_number=4, tag_tid=1, entry_tid=7024], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:54.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:54.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:54.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:54.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=7026' '!=' 'object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:54.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:54.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:45:54.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_26 2024-04-23T21:45:54.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:45:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_26 ']' 2024-04-23T21:45:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_26' 2024-04-23T21:45:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_26 2024-04-23T21:45:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:54.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:54.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:54.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_26 2024-04-23T21:45:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:54.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:54.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:54.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:54.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:54.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:54.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:54.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_26 2024-04-23T21:45:54.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:54.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_26 2024-04-23T21:45:54.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:54.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:54.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:54.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_26 '' 2024-04-23T21:45:54.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:54.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:54.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:54.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:54.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_26.status 2024-04-23T21:45:54.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_26.status 2024-04-23T21:45:54.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:54.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:54.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_26 2024-04-23T21:45:54.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:54.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:54.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:54.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:54.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=7026], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:54.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=7026], [object_number=5, tag_tid=1, entry_tid=7025], [object_number=4, tag_tid=1, entry_tid=7024], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:54.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:54.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:54.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_26 2024-04-23T21:45:54.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:54.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:54.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:54.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_26 '..*' 2024-04-23T21:45:54.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:54.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:54.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:54.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:54.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_26.status 2024-04-23T21:45:54.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_26.status 2024-04-23T21:45:54.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:54.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:54.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_26 2024-04-23T21:45:54.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:54.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:54.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:54.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:54.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=7026], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:54.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=7026], [object_number=5, tag_tid=1, entry_tid=7025], [object_number=4, tag_tid=1, entry_tid=7024], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:54.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:54.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:54.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:54.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=7026' '!=' 'object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:54.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:54.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:45:55.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_26 2024-04-23T21:45:55.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:55.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:55.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:45:55.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:55.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_26 ']' 2024-04-23T21:45:55.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_26' 2024-04-23T21:45:55.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_26 2024-04-23T21:45:55.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:55.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:55.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:55.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:55.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:55.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:55.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:55.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:55.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:55.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:55.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_26 2024-04-23T21:45:55.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:55.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:55.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:55.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:55.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:55.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:55.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:55.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:55.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:55.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:55.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:55.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:55.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:55.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:55.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:55.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:55.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:55.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:55.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:55.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:55.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:55.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:55.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:55.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:55.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:55.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:55.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:55.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:55.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:55.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:55.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:55.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:55.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:55.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_26 2024-04-23T21:45:55.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:55.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_26 2024-04-23T21:45:55.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:55.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:55.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:55.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_26 '' 2024-04-23T21:45:55.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:55.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:55.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:55.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:55.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_26.status 2024-04-23T21:45:55.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_26.status 2024-04-23T21:45:55.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:55.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:55.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_26 2024-04-23T21:45:55.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:55.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:55.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:55.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:55.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=7026], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:55.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=7026], [object_number=5, tag_tid=1, entry_tid=7025], [object_number=4, tag_tid=1, entry_tid=7024], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:55.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:55.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:55.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_26 2024-04-23T21:45:55.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:55.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:55.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:55.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_26 '..*' 2024-04-23T21:45:55.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_26.status 2024-04-23T21:45:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_26.status 2024-04-23T21:45:55.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:55.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:55.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_26 2024-04-23T21:45:55.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:55.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:55.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:55.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:55.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=7026], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:55.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=7026], [object_number=5, tag_tid=1, entry_tid=7025], [object_number=4, tag_tid=1, entry_tid=7024], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:55.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:55.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:55.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:55.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=7026' '!=' 'object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:55.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:55.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:45:57.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_26 2024-04-23T21:45:57.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:57.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:57.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:45:57.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:57.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_26 ']' 2024-04-23T21:45:57.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_26' 2024-04-23T21:45:57.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_26 2024-04-23T21:45:57.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:57.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:57.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:57.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:57.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:57.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:57.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:57.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:57.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:57.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:57.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_26 2024-04-23T21:45:57.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:57.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:57.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:57.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:57.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:57.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:57.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:57.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:57.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:57.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:57.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:57.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:57.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:57.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:57.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:57.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:57.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:57.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:57.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:57.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:57.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:57.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:57.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:57.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:57.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:57.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:57.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:57.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:57.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:57.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:57.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:57.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:57.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:57.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_26 2024-04-23T21:45:57.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:57.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_26 2024-04-23T21:45:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_26 '' 2024-04-23T21:45:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_26.status 2024-04-23T21:45:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_26.status 2024-04-23T21:45:57.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:57.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:57.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_26 2024-04-23T21:45:57.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:57.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:57.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:57.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:57.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=7026], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:57.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=7026], [object_number=5, tag_tid=1, entry_tid=7025], [object_number=4, tag_tid=1, entry_tid=7024], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:57.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:57.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:57.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_26 2024-04-23T21:45:57.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:57.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:57.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:57.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_26 '..*' 2024-04-23T21:45:57.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:57.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:57.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:57.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:57.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_26.status 2024-04-23T21:45:57.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_26.status 2024-04-23T21:45:57.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:57.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:57.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_26 2024-04-23T21:45:57.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:57.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:57.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:57.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:57.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=7026], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:57.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=7026], [object_number=5, tag_tid=1, entry_tid=7025], [object_number=4, tag_tid=1, entry_tid=7024], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:57.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:57.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:57.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:57.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=7026' '!=' 'object_number=6, tag_tid=1, entry_tid=7026' 2024-04-23T21:45:57.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:45:57.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:45:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_26 2024-04-23T21:45:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:45:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:45:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:45:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_26 ']' 2024-04-23T21:45:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_26' 2024-04-23T21:45:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_26 2024-04-23T21:45:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:45:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:59.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:45:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:45:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:45:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:45:59.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:45:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:45:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:45:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:45:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_26 2024-04-23T21:45:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:45:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:45:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:45:59.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:45:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:45:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:45:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:45:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:45:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:45:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:45:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:45:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:45:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:45:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:45:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:45:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:45:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:45:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:45:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:45:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:45:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:45:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:45:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:45:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:45:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:45:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:45:59.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:45:59.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:59.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:59.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:45:59.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_26 2024-04-23T21:45:59.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:45:59.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_26 2024-04-23T21:45:59.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:59.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:59.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:59.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_26 '' 2024-04-23T21:45:59.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:59.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:59.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:59.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:45:59.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_26.status 2024-04-23T21:45:59.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_26.status 2024-04-23T21:45:59.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:59.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:59.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_26 2024-04-23T21:45:59.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:59.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:45:59.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:45:59.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:45:59.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=7026], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:59.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=7026], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:45:59.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:59.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:45:59.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_26 2024-04-23T21:45:59.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:59.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:59.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:59.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_26 '..*' 2024-04-23T21:45:59.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:45:59.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:45:59.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:45:59.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:45:59.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_26.status 2024-04-23T21:45:59.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_26.status 2024-04-23T21:45:59.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:45:59.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:45:59.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_26 2024-04-23T21:45:59.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:46:00.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:46:00.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:46:00.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:00.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=7026], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:46:00.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=7026], [object_number=7, tag_tid=1, entry_tid=7023]]], state=connected] 2024-04-23T21:46:00.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.status 2024-04-23T21:46:00.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:46:00.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:46:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_26 snap 2024-04-23T21:46:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:46:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:46:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:46:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:46:00.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:46:01.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_26@snap 2024-04-23T21:46:01.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_26': 2024-04-23T21:46:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:46:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:46:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:46:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 116ea01a119e 2024-04-23T21:46:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.116ea01a119e 2024-04-23T21:46:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:46:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:46:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:46:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:46:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:35:58 2024 2024-04-23T21:46:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:35:58 2024 2024-04-23T21:46:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:36:59 2024 2024-04-23T21:46:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:46:01.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 116ea01a119e 2024-04-23T21:46:01.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:46:01.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:46:01.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: ef4cdba8-fa7e-4c68-a518-af82a968b35a 2024-04-23T21:46:01.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:46:01.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:01.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_26 snap 2024-04-23T21:46:01.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:01.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:46:01.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:46:01.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:46:01.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.export 2024-04-23T21:46:01.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_26.export 2024-04-23T21:46:01.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_26.export 2024-04-23T21:46:01.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_26@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.export 2024-04-23T21:46:01.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:46:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_26@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_26.export 2024-04-23T21:46:01.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:46:01.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_26.export 2024-04-23T21:46:01.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_26.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_26.export 2024-04-23T21:46:01.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:01.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:46:01.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_27 2024-04-23T21:46:01.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_27 snap 2024-04-23T21:46:01.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:46:01.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:01.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:46:01.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:46:01.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_27@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:46:03.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_27 2024-04-23T21:46:03.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:03.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:03.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:46:03.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_27 started 2024-04-23T21:46:03.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:03.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:03.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:46:03.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:46:03.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:46:03.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:03.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:46:04.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_27 started 2024-04-23T21:46:04.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:04.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:04.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:46:04.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:46:04.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:46:04.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:46:04.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_27 2024-04-23T21:46:04.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:46:04.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:46:04.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:04.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:04.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:46:04.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:46:04.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:04.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:46:04.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:04.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:46:04.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:04.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_27 2024-04-23T21:46:04.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:46:04.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:04.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:04.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:04.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:04.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:04.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:04.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:04.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:04.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:04.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:04.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:04.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:04.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:04.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:04.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:46:04.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:04.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:04.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:04.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:46:04.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:04.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:04.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:04.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:04.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:04.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:04.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:04.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:46:04.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:46:04.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:04.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:04.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:04.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:04.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_27 2024-04-23T21:46:04.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_27'] 2024-04-23T21:46:04.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:04.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:04.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:04.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_27 2024-04-23T21:46:04.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:46:04.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:04.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:04.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:04.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:04.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:04.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:04.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:04.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:04.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:46:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:46:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:04.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:04.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:04.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:04.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:04.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:46:04.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:46:04.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:04.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:04.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:04.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:04.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_27 2024-04-23T21:46:04.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_27'] 2024-04-23T21:46:04.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:04.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:04.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:04.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_27 2024-04-23T21:46:04.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:46:04.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:04.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:04.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:46:04.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:46:04.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:04.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:04.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:04.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:04.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:04.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:04.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:04.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:46:04.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:04.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:46:04.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:46:04.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:04.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:04.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:46:04.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:46:04.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:04.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:04.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:04.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:04.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:04.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:04.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:04.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:46:04.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:46:04.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:46:04.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:04.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:04.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:04.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_27 2024-04-23T21:46:04.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:46:04.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:46:04.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:46:04.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:46:04.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:46:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:46:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:46:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_27 2024-04-23T21:46:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:46:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:46:04.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:46:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:46:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_27 2024-04-23T21:46:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:46:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:46:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:46:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:46:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:46:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:46:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:46:04.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:46:04.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_27 2024-04-23T21:46:04.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:04.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:04.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:46:04.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:46:04.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_27 ']' 2024-04-23T21:46:04.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_27' 2024-04-23T21:46:04.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_27 2024-04-23T21:46:04.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:46:04.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:04.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:04.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:46:04.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:46:04.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:04.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:46:04.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:04.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:46:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_27 2024-04-23T21:46:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:46:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:46:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:46:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:04.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:46:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:46:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:04.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:04.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:04.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:04.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:04.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:04.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:04.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_27 2024-04-23T21:46:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_27'] 2024-04-23T21:46:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:05.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:05.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_27 2024-04-23T21:46:05.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:46:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:46:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:46:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:46:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:46:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:05.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:05.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:05.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:05.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:05.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:05.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_27 2024-04-23T21:46:05.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_27'] 2024-04-23T21:46:05.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:05.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:05.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:05.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_27 2024-04-23T21:46:05.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:05.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:46:05.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:46:05.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:05.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:05.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:05.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:05.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:05.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:05.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:05.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:46:05.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:05.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:46:05.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:05.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:05.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:05.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_27 2024-04-23T21:46:05.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:05.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_27 2024-04-23T21:46:05.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:05.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:05.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_27 2024-04-23T21:46:05.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_27 '' 2024-04-23T21:46:05.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:05.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:05.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_27 2024-04-23T21:46:05.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:46:05.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_27.status 2024-04-23T21:46:05.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_27.status 2024-04-23T21:46:05.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:05.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_27.status 2024-04-23T21:46:05.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_27 2024-04-23T21:46:05.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_27.status 2024-04-23T21:46:05.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:46:05.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:46:05.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:05.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13499], [object_number=14, tag_tid=1, entry_tid=13498]]], state=connected] 2024-04-23T21:46:05.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13499], [object_number=14, tag_tid=1, entry_tid=13498]]], state=connected] 2024-04-23T21:46:05.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_27.status 2024-04-23T21:46:05.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:46:05.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_27 2024-04-23T21:46:05.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:05.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:05.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_27 2024-04-23T21:46:05.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_27 '..*' 2024-04-23T21:46:05.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_27 2024-04-23T21:46:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:46:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_27.status 2024-04-23T21:46:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_27.status 2024-04-23T21:46:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_27.status 2024-04-23T21:46:05.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_27 2024-04-23T21:46:05.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_27.status 2024-04-23T21:46:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:46:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:46:05.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13499], [object_number=14, tag_tid=1, entry_tid=13498]]], state=connected] 2024-04-23T21:46:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13499], [object_number=14, tag_tid=1, entry_tid=13498]]], state=connected] 2024-04-23T21:46:05.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_27.status 2024-04-23T21:46:05.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:46:05.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:46:05.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_27 snap 2024-04-23T21:46:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:46:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:46:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:46:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:46:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:46:06.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_27@snap 2024-04-23T21:46:06.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_27': 2024-04-23T21:46:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:46:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:46:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:46:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 11b8735d0266 2024-04-23T21:46:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.11b8735d0266 2024-04-23T21:46:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:46:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:46:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:46:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:46:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:36:06 2024 2024-04-23T21:46:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:36:06 2024 2024-04-23T21:46:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:38:08 2024 2024-04-23T21:46:06.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:46:06.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 11b8735d0266 2024-04-23T21:46:06.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:46:06.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:46:06.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 32be9fcc-7049-47d4-813d-6edb9a876f43 2024-04-23T21:46:06.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:46:06.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:06.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_27 snap 2024-04-23T21:46:06.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:06.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:46:06.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:46:06.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:46:06.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_27.export 2024-04-23T21:46:06.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_27.export 2024-04-23T21:46:06.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_27.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_27.export 2024-04-23T21:46:06.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_27@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_27.export 2024-04-23T21:46:06.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:46:06.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_27@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_27.export 2024-04-23T21:46:07.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:46:07.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_27.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_27.export 2024-04-23T21:46:07.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_27.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_27.export 2024-04-23T21:46:07.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:07.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:46:07.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_28 2024-04-23T21:46:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_28 snap 2024-04-23T21:46:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:46:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:46:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:46:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_28@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:46:09.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_28 2024-04-23T21:46:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:46:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_28 started 2024-04-23T21:46:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:46:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:46:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:46:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:09.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:46:10.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_28 started 2024-04-23T21:46:10.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:10.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:10.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:46:10.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:46:10.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:46:10.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:46:10.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_28 2024-04-23T21:46:10.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:46:10.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:10.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:10.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:46:10.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:46:10.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:46:10.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:10.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:46:10.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:10.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:46:10.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:10.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_28 2024-04-23T21:46:10.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:46:10.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:10.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:10.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:10.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:10.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:10.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:10.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:10.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:10.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:10.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:10.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:10.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:10.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:10.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:10.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:46:10.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:10.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:10.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:10.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:46:10.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:10.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:10.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:10.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:10.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:10.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:10.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:10.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:46:10.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:46:10.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:10.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:10.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:10.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:10.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_28 2024-04-23T21:46:10.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_28'] 2024-04-23T21:46:10.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:10.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:10.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:10.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_28 2024-04-23T21:46:10.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:46:10.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:10.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:10.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:10.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:10.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:10.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:10.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:10.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:10.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:10.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:10.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:10.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:46:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:46:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:10.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:46:10.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:46:10.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:10.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:10.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:10.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:10.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_28 2024-04-23T21:46:10.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:46:10.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:46:10.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:46:10.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:46:10.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:46:10.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:46:10.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:46:10.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:10.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_28 2024-04-23T21:46:10.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:46:10.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:46:10.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:10.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:46:10.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:46:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_28 2024-04-23T21:46:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:46:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:46:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:46:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:46:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:46:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:46:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:46:10.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:46:10.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_28 2024-04-23T21:46:10.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:10.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:10.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:46:10.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:46:10.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_28 ']' 2024-04-23T21:46:10.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_28' 2024-04-23T21:46:10.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_28 2024-04-23T21:46:10.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:46:10.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:10.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:10.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:46:10.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:46:10.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:10.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:46:10.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:46:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_28 2024-04-23T21:46:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:46:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:46:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:46:10.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:10.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:10.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:10.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:46:10.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:10.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:10.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:46:10.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:10.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:10.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:10.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:10.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:10.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:10.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:10.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:10.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:10.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:10.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:10.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:10.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:10.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:10.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:10.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:10.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:10.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_28 2024-04-23T21:46:10.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_28'] 2024-04-23T21:46:10.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:10.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:10.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:10.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_28 2024-04-23T21:46:10.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:46:10.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:10.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:10.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:10.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:46:10.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:10.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:10.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:10.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:46:10.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:10.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:10.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:10.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:46:10.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:10.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:10.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:46:10.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:10.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:10.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:10.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:10.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:10.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:10.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:10.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:10.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:10.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:10.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:10.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:10.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:10.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:11.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:11.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:11.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:11.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_28 2024-04-23T21:46:11.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:11.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_28 2024-04-23T21:46:11.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:11.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:11.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:11.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_28 '' 2024-04-23T21:46:11.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:11.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:11.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:11.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:46:11.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_28.status 2024-04-23T21:46:11.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_28.status 2024-04-23T21:46:11.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:11.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:11.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_28 2024-04-23T21:46:11.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:11.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:46:11.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:46:11.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:11.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710]]], state=connected] 2024-04-23T21:46:11.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710], [object_number=9, tag_tid=1, entry_tid=11709], [object_number=8, tag_tid=1, entry_tid=11708]]], state=connected] 2024-04-23T21:46:11.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:11.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:46:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_28 2024-04-23T21:46:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:11.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_28 '..*' 2024-04-23T21:46:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:46:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_28.status 2024-04-23T21:46:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_28.status 2024-04-23T21:46:11.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:11.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:11.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_28 2024-04-23T21:46:11.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:11.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:46:11.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:46:11.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:11.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710]]], state=connected] 2024-04-23T21:46:11.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710], [object_number=9, tag_tid=1, entry_tid=11709], [object_number=8, tag_tid=1, entry_tid=11708]]], state=connected] 2024-04-23T21:46:11.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:11.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=11, tag_tid=1, entry_tid=11711' 2024-04-23T21:46:11.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=11, tag_tid=1, entry_tid=11711' 2024-04-23T21:46:11.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=11711' '!=' '' 2024-04-23T21:46:11.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:46:11.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=11711' = '' 2024-04-23T21:46:11.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=11, tag_tid=1, entry_tid=11711' 2024-04-23T21:46:11.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:46:11.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:46:11.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:46:11.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:46:11.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=11, tag_tid=1, entry_tid=11711' 2024-04-23T21:46:11.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:46:11.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:46:11.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:46:11.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:46:11.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:46:11.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:46:11.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:46:11.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=11, tag_tid=1, entry_tid=11711' 2024-04-23T21:46:11.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:46:11.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:46:11.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=11711 2024-04-23T21:46:11.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 11711 2024-04-23T21:46:11.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:46:11.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:46:11.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:46:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_28 2024-04-23T21:46:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:46:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:46:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_28 ']' 2024-04-23T21:46:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_28' 2024-04-23T21:46:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_28 2024-04-23T21:46:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:46:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:46:11.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:46:11.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:11.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:46:11.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:11.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:46:11.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:11.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_28 2024-04-23T21:46:11.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:46:11.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:11.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:11.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:11.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:46:11.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:11.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:11.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:11.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:46:11.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:11.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:11.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:11.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:46:11.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:11.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:11.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:46:11.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:11.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:11.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:11.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:11.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:11.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:11.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:11.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:11.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:11.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:11.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:11.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:11.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:11.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:11.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:11.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:11.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:11.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_28 2024-04-23T21:46:11.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_28'] 2024-04-23T21:46:11.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:11.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:11.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:11.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_28 2024-04-23T21:46:11.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:46:11.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:11.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:11.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:11.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:46:11.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:11.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:11.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:11.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:46:11.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:11.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:11.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:11.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:46:11.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:11.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:11.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:46:11.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:11.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:11.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:11.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:11.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:11.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:11.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:11.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:11.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:11.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:11.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:11.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:11.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:11.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:11.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:11.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:11.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:11.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_28 2024-04-23T21:46:11.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:11.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_28 2024-04-23T21:46:11.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:11.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:11.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:11.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_28 '' 2024-04-23T21:46:11.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:11.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:11.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:11.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:46:11.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_28.status 2024-04-23T21:46:11.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_28.status 2024-04-23T21:46:11.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:11.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:11.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_28 2024-04-23T21:46:11.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:11.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:46:11.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:46:11.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:11.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710]]], state=connected] 2024-04-23T21:46:11.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710], [object_number=9, tag_tid=1, entry_tid=11709], [object_number=8, tag_tid=1, entry_tid=11708]]], state=connected] 2024-04-23T21:46:11.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:11.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:46:11.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_28 2024-04-23T21:46:11.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:11.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:11.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:11.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_28 '..*' 2024-04-23T21:46:11.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:11.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:11.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:11.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:46:11.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_28.status 2024-04-23T21:46:11.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_28.status 2024-04-23T21:46:11.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:11.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:11.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_28 2024-04-23T21:46:11.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:11.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:46:11.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:46:11.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:11.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710]]], state=connected] 2024-04-23T21:46:11.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710], [object_number=9, tag_tid=1, entry_tid=11709], [object_number=8, tag_tid=1, entry_tid=11708]]], state=connected] 2024-04-23T21:46:11.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:11.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=11, tag_tid=1, entry_tid=11711' 2024-04-23T21:46:11.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=11, tag_tid=1, entry_tid=11711' 2024-04-23T21:46:11.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=11711' '!=' 'object_number=11, tag_tid=1, entry_tid=11711' 2024-04-23T21:46:11.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:46:11.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:46:12.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_28 2024-04-23T21:46:12.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:12.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:12.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:46:12.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:46:12.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_28 ']' 2024-04-23T21:46:12.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_28' 2024-04-23T21:46:12.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_28 2024-04-23T21:46:12.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:46:12.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:12.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:12.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:46:12.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:46:12.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:12.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:46:12.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:12.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:46:12.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:12.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_28 2024-04-23T21:46:12.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:46:12.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:12.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:12.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:12.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:46:12.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:12.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:12.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:12.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:46:12.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:12.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:12.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:12.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:46:12.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:12.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:12.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:46:12.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:12.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:12.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:12.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:12.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:12.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:12.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:12.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:12.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:12.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:12.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:12.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:12.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:12.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:12.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:12.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:12.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:12.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_28 2024-04-23T21:46:12.143 INFO:tasks.ceph.cluster1.osd.2.smithi148.stderr:2024-04-23T21:46:12.144+0000 7f9212057640 -1 reset not still connected to 0x5614da203c70 2024-04-23T21:46:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_28'] 2024-04-23T21:46:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:12.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:12.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:12.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_28 2024-04-23T21:46:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:46:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:46:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:46:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:46:12.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:12.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:12.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:46:12.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:12.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:12.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:12.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:12.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:12.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:12.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:12.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:12.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:12.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:12.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:12.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:12.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:12.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:12.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:12.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:12.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:12.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_28 2024-04-23T21:46:12.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:12.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_28 2024-04-23T21:46:12.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:12.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:12.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:12.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_28 '' 2024-04-23T21:46:12.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:12.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:12.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:12.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:46:12.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_28.status 2024-04-23T21:46:12.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_28.status 2024-04-23T21:46:12.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:12.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:12.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_28 2024-04-23T21:46:12.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:12.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:46:12.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:46:12.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:12.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710]]], state=connected] 2024-04-23T21:46:12.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710], [object_number=9, tag_tid=1, entry_tid=11709], [object_number=8, tag_tid=1, entry_tid=11708]]], state=connected] 2024-04-23T21:46:12.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:12.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:46:12.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_28 2024-04-23T21:46:12.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:12.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:12.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_28 '..*' 2024-04-23T21:46:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:46:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_28.status 2024-04-23T21:46:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_28.status 2024-04-23T21:46:12.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:12.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:12.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_28 2024-04-23T21:46:12.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:12.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:46:12.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:46:12.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:12.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710]]], state=connected] 2024-04-23T21:46:12.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710], [object_number=9, tag_tid=1, entry_tid=11709], [object_number=8, tag_tid=1, entry_tid=11708]]], state=connected] 2024-04-23T21:46:12.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:12.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=11, tag_tid=1, entry_tid=11711' 2024-04-23T21:46:12.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=11, tag_tid=1, entry_tid=11711' 2024-04-23T21:46:12.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=11711' '!=' 'object_number=11, tag_tid=1, entry_tid=11711' 2024-04-23T21:46:12.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:46:12.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:46:13.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_28 2024-04-23T21:46:13.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:13.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:13.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:46:13.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:46:13.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_28 ']' 2024-04-23T21:46:13.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_28' 2024-04-23T21:46:13.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_28 2024-04-23T21:46:13.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:46:13.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:13.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:13.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:46:13.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:46:13.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:13.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:46:13.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:13.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:46:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_28 2024-04-23T21:46:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:46:13.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:13.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:13.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:13.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:46:13.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:13.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:13.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:13.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:46:13.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:13.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:13.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:13.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:46:13.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:13.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:13.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:46:13.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:13.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:13.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:13.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:13.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:13.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:13.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:13.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:13.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:13.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:13.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:13.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:13.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:13.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:13.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:13.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:13.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:13.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_28 2024-04-23T21:46:13.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_28'] 2024-04-23T21:46:13.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:13.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:13.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:13.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_28 2024-04-23T21:46:13.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:46:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:46:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:46:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:46:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:13.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:46:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:13.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:13.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:13.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:13.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:13.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_28 2024-04-23T21:46:13.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:13.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_28 2024-04-23T21:46:13.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:13.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:13.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:13.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_28 '' 2024-04-23T21:46:13.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:13.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:13.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:13.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:46:13.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_28.status 2024-04-23T21:46:13.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_28.status 2024-04-23T21:46:13.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:13.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:13.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_28 2024-04-23T21:46:13.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:13.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:46:13.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:46:13.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:13.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710]]], state=connected] 2024-04-23T21:46:13.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710], [object_number=9, tag_tid=1, entry_tid=11709], [object_number=8, tag_tid=1, entry_tid=11708]]], state=connected] 2024-04-23T21:46:13.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:13.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:46:13.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_28 2024-04-23T21:46:13.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:13.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:13.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:13.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_28 '..*' 2024-04-23T21:46:13.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:13.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:13.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:13.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:46:13.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_28.status 2024-04-23T21:46:13.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_28.status 2024-04-23T21:46:13.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:13.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:13.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_28 2024-04-23T21:46:13.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:13.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:46:13.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:46:13.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:13.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710]]], state=connected] 2024-04-23T21:46:13.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710], [object_number=9, tag_tid=1, entry_tid=11709], [object_number=8, tag_tid=1, entry_tid=11708]]], state=connected] 2024-04-23T21:46:13.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:13.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=11, tag_tid=1, entry_tid=11711' 2024-04-23T21:46:13.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=11, tag_tid=1, entry_tid=11711' 2024-04-23T21:46:13.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=11711' '!=' 'object_number=11, tag_tid=1, entry_tid=11711' 2024-04-23T21:46:13.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:46:13.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:46:15.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_28 2024-04-23T21:46:15.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:15.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:15.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:46:15.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:46:15.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_28 ']' 2024-04-23T21:46:15.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_28' 2024-04-23T21:46:15.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_28 2024-04-23T21:46:15.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:46:15.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:15.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:15.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:46:15.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:46:15.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:15.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:46:15.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:15.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:46:15.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:15.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_28 2024-04-23T21:46:15.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:46:15.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:15.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:15.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:15.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:46:15.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:15.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:15.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:15.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:46:15.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:15.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:15.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:15.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:46:15.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:15.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:15.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:46:15.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:15.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:15.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:15.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:15.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:15.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:15.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:15.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:15.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:15.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:15.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:15.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:15.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:15.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:15.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:15.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:15.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:15.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_28 2024-04-23T21:46:15.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_28'] 2024-04-23T21:46:15.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:15.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:15.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:15.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_28 2024-04-23T21:46:15.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:46:15.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:15.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:15.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:15.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:46:15.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:15.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:15.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:15.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:46:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:46:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:46:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:15.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:15.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:15.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:15.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:15.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:15.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:15.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:15.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:15.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:15.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:15.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:15.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_28 2024-04-23T21:46:15.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:15.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_28 2024-04-23T21:46:15.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:15.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:15.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:15.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_28 '' 2024-04-23T21:46:15.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:15.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:15.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:15.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:46:15.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_28.status 2024-04-23T21:46:15.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_28.status 2024-04-23T21:46:15.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:15.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:15.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_28 2024-04-23T21:46:15.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:15.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:46:15.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:46:15.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:15.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710]]], state=connected] 2024-04-23T21:46:15.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710]]], state=connected] 2024-04-23T21:46:15.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:15.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:46:15.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_28 2024-04-23T21:46:15.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:15.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:15.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:15.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_28 '..*' 2024-04-23T21:46:15.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:15.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:15.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:46:15.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:46:15.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_28.status 2024-04-23T21:46:15.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_28.status 2024-04-23T21:46:15.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_28 2024-04-23T21:46:15.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:15.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:46:15.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:46:15.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:15.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710]]], state=connected] 2024-04-23T21:46:15.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=11711], [object_number=10, tag_tid=1, entry_tid=11710]]], state=connected] 2024-04-23T21:46:15.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.status 2024-04-23T21:46:15.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:46:15.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:46:15.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:15.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_28 snap 2024-04-23T21:46:15.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:15.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:15.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:46:15.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:46:15.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:46:15.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:46:15.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:46:16.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_28@snap 2024-04-23T21:46:16.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_28': 2024-04-23T21:46:16.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:46:16.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:46:16.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:46:16.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 121db344050d 2024-04-23T21:46:16.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.121db344050d 2024-04-23T21:46:16.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:46:16.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:46:16.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:46:16.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:46:16.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:36:19 2024 2024-04-23T21:46:16.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:36:19 2024 2024-04-23T21:46:16.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:37:20 2024 2024-04-23T21:46:16.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:46:16.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 121db344050d 2024-04-23T21:46:16.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:46:16.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:46:16.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: ccc35c80-c5d9-4834-950e-bf55c60243d6 2024-04-23T21:46:16.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:46:16.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:16.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_28 snap 2024-04-23T21:46:16.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:16.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:46:16.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:46:16.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:46:16.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.export 2024-04-23T21:46:16.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_28.export 2024-04-23T21:46:16.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_28.export 2024-04-23T21:46:16.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_28@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.export 2024-04-23T21:46:16.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:46:16.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_28@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_28.export 2024-04-23T21:46:16.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:46:16.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_28.export 2024-04-23T21:46:16.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_28.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_28.export 2024-04-23T21:46:16.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:16.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:46:16.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_29 2024-04-23T21:46:16.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_29 snap 2024-04-23T21:46:16.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:46:16.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:16.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:46:16.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:46:16.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_29@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:46:17.451 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:46:17.451+0000 7f83ebb59640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 2 2024-04-23T21:46:17.451 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:46:17.452+0000 7f83ed35c640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 6 2024-04-23T21:46:18.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_29 2024-04-23T21:46:18.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:18.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:18.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:46:18.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_29 started 2024-04-23T21:46:18.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:18.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:18.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:46:18.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:46:18.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:46:18.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:18.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:46:19.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_29 started 2024-04-23T21:46:19.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:19.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:19.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:46:19.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:46:19.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:46:19.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:46:19.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_29 2024-04-23T21:46:19.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:46:19.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:19.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:46:19.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:19.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:46:19.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:46:19.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:19.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:46:19.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:19.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:46:19.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:19.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_29 2024-04-23T21:46:19.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:46:19.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:19.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:19.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:19.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:19.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:19.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:19.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:19.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:19.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:19.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:19.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:19.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:19.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:19.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:19.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:46:19.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:19.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:19.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:19.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:46:19.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:19.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:19.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:19.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:19.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:19.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:19.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:19.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:46:19.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:46:19.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:19.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:19.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:19.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:19.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_29 2024-04-23T21:46:19.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:19.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:19.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:19.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:19.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_29 2024-04-23T21:46:19.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:46:19.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:19.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:19.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:19.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:19.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:19.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:19.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:19.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:19.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:19.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:19.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:19.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:19.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:46:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:46:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:46:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:46:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:19.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:19.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:19.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:19.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_29 2024-04-23T21:46:19.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:19.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:19.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:19.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:19.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_29 2024-04-23T21:46:19.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:46:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:46:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:46:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:46:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:19.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:46:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:46:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:46:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:46:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:46:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:46:19.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:46:19.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:19.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:19.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:19.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_29 2024-04-23T21:46:19.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:19.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:19.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:19.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:19.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_29 2024-04-23T21:46:19.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:46:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:46:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:46:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:46:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:46:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:46:19.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:19.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:19.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:46:19.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:46:19.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:19.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:19.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:19.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:19.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:19.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:19.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:19.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:46:19.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:46:19.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:46:19.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:19.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:19.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:19.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_29 2024-04-23T21:46:19.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:46:19.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:46:19.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:46:19.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:46:19.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:46:19.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:46:19.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:46:19.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:19.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_29 2024-04-23T21:46:19.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:46:19.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:46:19.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:19.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:46:19.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:46:19.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_29 2024-04-23T21:46:19.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:46:19.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:46:19.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:19.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:46:19.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:46:19.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:46:19.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:46:19.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:46:19.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:46:20.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_29 2024-04-23T21:46:20.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:20.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:20.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:46:20.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:46:20.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_29 ']' 2024-04-23T21:46:20.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_29' 2024-04-23T21:46:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_29 2024-04-23T21:46:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:46:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:46:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:46:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:46:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:20.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:46:20.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_29 2024-04-23T21:46:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:46:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:46:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:46:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:46:20.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:20.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:20.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:46:20.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:20.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:20.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:20.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:20.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:20.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:20.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:20.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:20.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:20.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:20.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:20.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:20.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:20.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:20.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:20.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:20.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:20.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:20.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:20.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:20.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:20.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_29 2024-04-23T21:46:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:46:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:46:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:46:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:20.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:46:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:46:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:20.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:20.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:20.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:20.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:20.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:20.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:20.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:20.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:20.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:20.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:20.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:20.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:20.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_29 2024-04-23T21:46:20.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:46:20.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:20.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:20.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:46:20.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:46:20.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:20.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:20.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:20.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:46:20.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:20.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:20.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:20.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:46:20.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:20.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:46:20.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:46:20.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:20.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:20.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:46:20.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:46:20.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:20.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:20.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:20.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:20.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:20.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:20.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:20.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:46:20.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:20.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:46:20.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:20.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:20.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:20.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:20.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:20.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:20.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_29 2024-04-23T21:46:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:46:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:46:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:46:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:46:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:20.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:46:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:46:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:46:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:46:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:46:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:20.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:20.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:20.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:46:20.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:20.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:46:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:20.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:20.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_29 2024-04-23T21:46:20.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:20.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:20.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:20.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_29 '' 2024-04-23T21:46:20.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:20.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:20.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:20.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:46:20.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_29.status 2024-04-23T21:46:20.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_29.status 2024-04-23T21:46:20.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:20.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:20.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_29 2024-04-23T21:46:20.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:20.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:46:20.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:46:20.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:20.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8306], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:20.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=8306], [object_number=5, tag_tid=1, entry_tid=8305], [object_number=4, tag_tid=1, entry_tid=8304], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:20.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:20.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:46:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_29 2024-04-23T21:46:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_29 '..*' 2024-04-23T21:46:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:46:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_29.status 2024-04-23T21:46:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_29.status 2024-04-23T21:46:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_29 2024-04-23T21:46:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:20.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:46:20.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:46:20.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:20.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8306], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:20.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=8306], [object_number=5, tag_tid=1, entry_tid=8305], [object_number=4, tag_tid=1, entry_tid=8304], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:20.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:20.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:20.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:20.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=8306' '!=' '' 2024-04-23T21:46:20.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:46:20.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=8306' = '' 2024-04-23T21:46:20.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:20.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:46:20.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:46:20.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:46:20.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:46:20.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:20.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:46:20.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:46:20.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:46:20.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:46:20.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:46:20.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:46:20.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:46:20.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:20.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:46:20.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:46:20.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=8306 2024-04-23T21:46:20.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 8306 2024-04-23T21:46:20.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:46:20.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:46:20.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:46:20.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_29 2024-04-23T21:46:20.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:20.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:20.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:46:20.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:46:20.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_29 ']' 2024-04-23T21:46:20.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_29' 2024-04-23T21:46:20.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_29 2024-04-23T21:46:20.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:46:20.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:20.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:20.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:46:20.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:46:20.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:20.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:46:20.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:20.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:46:20.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:20.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_29 2024-04-23T21:46:20.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:46:20.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:20.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:20.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:20.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:46:20.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:20.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:20.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:46:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:46:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:46:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:20.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:20.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:20.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:20.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:20.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:20.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:20.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:20.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:20.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:20.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:20.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:20.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:20.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:20.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:20.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:20.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:20.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_29 2024-04-23T21:46:20.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:46:20.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:46:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:46:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:46:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:20.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:46:20.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:20.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:20.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:20.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:20.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:20.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:20.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:20.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:20.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:20.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:20.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:20.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:20.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:20.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:20.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:20.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:20.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:20.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:20.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:20.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:20.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:20.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:20.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_29 2024-04-23T21:46:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:46:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:46:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:46:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:46:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:46:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:20.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:46:21.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:46:21.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:21.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:21.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:46:21.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:46:21.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:21.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:21.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:21.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:21.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:21.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:21.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:21.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:46:21.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:21.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:46:21.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:21.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:21.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:21.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:21.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:21.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:21.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:21.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:21.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_29 2024-04-23T21:46:21.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:46:21.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:21.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:21.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:46:21.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:46:21.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:21.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:21.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:21.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:21.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:21.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:21.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:46:21.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:21.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:46:21.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:21.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:21.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:21.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:21.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_29 2024-04-23T21:46:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_29 '' 2024-04-23T21:46:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:21.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:46:21.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_29.status 2024-04-23T21:46:21.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_29.status 2024-04-23T21:46:21.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:21.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:21.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_29 2024-04-23T21:46:21.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:21.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:46:21.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:46:21.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:21.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8306], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:21.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=8306], [object_number=5, tag_tid=1, entry_tid=8305], [object_number=4, tag_tid=1, entry_tid=8304], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:21.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:21.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:46:21.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_29 2024-04-23T21:46:21.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:21.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:21.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:21.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_29 '..*' 2024-04-23T21:46:21.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:21.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:21.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:21.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:46:21.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_29.status 2024-04-23T21:46:21.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_29.status 2024-04-23T21:46:21.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:21.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:21.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_29 2024-04-23T21:46:21.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:21.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:46:21.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:46:21.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:21.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8306], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:21.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=8306], [object_number=5, tag_tid=1, entry_tid=8305], [object_number=4, tag_tid=1, entry_tid=8304], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:21.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:21.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:21.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:21.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=8306' '!=' 'object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:21.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:46:21.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:46:21.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_29 2024-04-23T21:46:21.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:21.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:21.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:46:21.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:46:21.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_29 ']' 2024-04-23T21:46:21.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_29' 2024-04-23T21:46:21.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_29 2024-04-23T21:46:21.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:46:21.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:21.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:21.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:46:21.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:46:21.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:21.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:46:21.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:21.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:46:21.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:21.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_29 2024-04-23T21:46:21.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:46:21.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:21.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:21.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:21.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:46:21.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:21.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:21.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:21.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:21.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:21.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:21.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:21.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:21.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:21.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:21.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:21.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:21.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:21.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:21.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:21.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:21.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_29 2024-04-23T21:46:21.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:46:21.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:21.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:21.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:21.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:46:21.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:46:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:46:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:46:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:21.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:21.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:21.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:21.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:21.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:21.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:21.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:21.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:21.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:21.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:21.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:21.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:21.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:21.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:21.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:21.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:21.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:21.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_29 2024-04-23T21:46:21.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:46:21.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:21.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:21.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:46:21.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:46:21.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:21.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:21.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:46:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:46:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:46:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:46:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:46:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:46:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:21.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:21.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:21.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:21.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:21.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:46:21.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:21.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:46:21.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:21.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:21.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:21.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:22.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:22.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:22.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:22.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:22.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_29 2024-04-23T21:46:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:46:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:46:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:46:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:46:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:46:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:22.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:46:22.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:46:22.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:22.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:22.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:46:22.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:46:22.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:22.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:22.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:22.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:22.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:22.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:22.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:22.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:46:22.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:22.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:46:22.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:22.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:22.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:22.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:22.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_29 2024-04-23T21:46:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_29 '' 2024-04-23T21:46:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:22.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:46:22.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_29.status 2024-04-23T21:46:22.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_29.status 2024-04-23T21:46:22.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:22.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:22.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_29 2024-04-23T21:46:22.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:22.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:46:22.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:46:22.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:22.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8306], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:22.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=8306], [object_number=5, tag_tid=1, entry_tid=8305], [object_number=4, tag_tid=1, entry_tid=8304], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:22.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:22.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:46:22.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_29 2024-04-23T21:46:22.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:22.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:22.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:22.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_29 '..*' 2024-04-23T21:46:22.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:22.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:22.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:22.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:46:22.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_29.status 2024-04-23T21:46:22.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_29.status 2024-04-23T21:46:22.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:22.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:22.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_29 2024-04-23T21:46:22.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:22.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:46:22.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:46:22.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:22.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8306], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:22.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=8306], [object_number=5, tag_tid=1, entry_tid=8305], [object_number=4, tag_tid=1, entry_tid=8304], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:22.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:22.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=8306' '!=' 'object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:22.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:46:22.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:46:23.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_29 2024-04-23T21:46:23.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:23.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:23.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:46:23.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:46:23.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_29 ']' 2024-04-23T21:46:23.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_29' 2024-04-23T21:46:23.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_29 2024-04-23T21:46:23.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:46:23.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:23.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:23.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:46:23.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:46:23.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:23.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:46:23.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:23.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:46:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_29 2024-04-23T21:46:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:46:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:46:23.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:23.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:23.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:23.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:23.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:23.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:23.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:23.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:23.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:23.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:23.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:23.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:23.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:23.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:23.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:23.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:23.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:23.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_29 2024-04-23T21:46:23.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:46:23.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:23.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:23.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:23.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:46:23.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:23.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:23.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:23.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:46:23.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:23.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:23.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:23.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:46:23.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:23.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:23.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:46:23.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:23.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:23.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:23.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:23.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:23.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:23.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:23.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:23.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:23.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:23.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:23.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:23.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:23.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:23.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:23.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:23.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:23.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:23.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:23.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:23.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:23.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:23.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_29 2024-04-23T21:46:23.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:46:23.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:23.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:23.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:46:23.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:46:23.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:23.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:23.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:23.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:46:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:46:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:46:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:46:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:46:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:46:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:23.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:23.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:23.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:23.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:23.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:46:23.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:23.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:46:23.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:23.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:23.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:23.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:23.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:23.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:23.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:23.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:23.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_29 2024-04-23T21:46:23.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:46:23.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:23.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:23.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:46:23.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:46:23.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:46:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:46:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:46:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:46:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:46:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:46:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:23.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:46:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:23.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:46:23.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:23.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:23.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:23.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:23.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:23.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_29 2024-04-23T21:46:23.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:23.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:23.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:23.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_29 '' 2024-04-23T21:46:23.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:23.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:23.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:23.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:46:23.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_29.status 2024-04-23T21:46:23.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_29.status 2024-04-23T21:46:23.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:23.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:23.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_29 2024-04-23T21:46:23.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:23.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:46:23.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:46:23.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:23.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8306], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:23.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=8306], [object_number=5, tag_tid=1, entry_tid=8305], [object_number=4, tag_tid=1, entry_tid=8304], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:23.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:23.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:46:23.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_29 2024-04-23T21:46:23.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:23.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:23.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:23.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_29 '..*' 2024-04-23T21:46:23.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:23.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:23.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:23.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:46:23.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_29.status 2024-04-23T21:46:23.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_29.status 2024-04-23T21:46:23.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:23.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:23.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_29 2024-04-23T21:46:23.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:23.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:46:23.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:46:23.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:23.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8306], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:23.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=8306], [object_number=5, tag_tid=1, entry_tid=8305], [object_number=4, tag_tid=1, entry_tid=8304], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:23.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:23.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:23.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:23.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=8306' '!=' 'object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:23.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:46:23.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:46:25.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_29 2024-04-23T21:46:25.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:25.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:25.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:46:25.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:46:25.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_29 ']' 2024-04-23T21:46:25.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_29' 2024-04-23T21:46:25.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_29 2024-04-23T21:46:25.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:46:25.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:25.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:25.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:46:25.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:46:25.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:25.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:46:25.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:25.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:46:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_29 2024-04-23T21:46:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:46:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:46:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:46:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:25.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:46:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:46:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:25.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:25.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:25.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:25.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:25.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:25.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:25.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:25.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:25.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:25.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:25.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:25.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_29 2024-04-23T21:46:25.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:46:25.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:25.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:25.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:25.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:46:25.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:46:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:46:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:46:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:25.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:25.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:25.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:25.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:25.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:25.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:25.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:25.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:25.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:25.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:25.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:25.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:25.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:25.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:25.294 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2024-04-23T21:46:25.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2024-04-23T21:46:25.294 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2024-04-23T21:46:25.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:46:25.294+0000 7f5e6275d640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 64486) UID: 1000 2024-04-23T21:46:25.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:25.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:25.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:25.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:25.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_29 2024-04-23T21:46:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:46:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:46:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:46:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:46:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:46:25.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:46:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:46:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:46:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:46:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:46:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:25.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:46:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:25.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:25.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:25.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:25.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:25.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_29 2024-04-23T21:46:25.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:46:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:46:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:46:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:46:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:46:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:25.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:46:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:46:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:46:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:46:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:46:25.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:25.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:46:25.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:25.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_29 2024-04-23T21:46:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_29 '' 2024-04-23T21:46:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:46:25.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_29.status 2024-04-23T21:46:25.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_29.status 2024-04-23T21:46:25.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:25.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:25.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_29 2024-04-23T21:46:25.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:25.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:46:25.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:46:25.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:25.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8306], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:25.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=8306], [object_number=5, tag_tid=1, entry_tid=8305], [object_number=4, tag_tid=1, entry_tid=8304], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:25.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:25.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:46:25.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_29 2024-04-23T21:46:25.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:25.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:25.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:25.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_29 '..*' 2024-04-23T21:46:25.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:25.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:25.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:25.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:46:25.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_29.status 2024-04-23T21:46:25.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_29.status 2024-04-23T21:46:25.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:25.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:25.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_29 2024-04-23T21:46:25.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:25.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:46:25.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:46:25.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:25.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8306], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:25.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=8306], [object_number=5, tag_tid=1, entry_tid=8305], [object_number=4, tag_tid=1, entry_tid=8304], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:25.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:25.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:25.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:25.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=8306' '!=' 'object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:25.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:46:25.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:46:27.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_29 2024-04-23T21:46:27.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:27.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:27.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:46:27.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:46:27.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_29 ']' 2024-04-23T21:46:27.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_29' 2024-04-23T21:46:27.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_29 2024-04-23T21:46:27.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:46:27.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:27.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:27.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:46:27.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:46:27.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:27.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:46:27.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:27.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:46:27.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:27.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_29 2024-04-23T21:46:27.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:46:27.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:27.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:27.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:27.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:46:27.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:27.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:27.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:27.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:46:27.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:27.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:27.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:27.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:46:27.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:27.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:27.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:46:27.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:27.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:27.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:27.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:27.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:27.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:27.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:27.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:27.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:27.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:27.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:27.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:27.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:27.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:27.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:27.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:27.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:27.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:27.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2024-04-23T21:46:27.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:27.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_29 2024-04-23T21:46:27.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:46:27.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:27.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:27.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:27.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:46:27.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:27.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:27.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:27.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:46:27.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:27.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:27.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:27.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:46:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:46:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:27.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:27.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:27.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:27.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:27.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:27.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:27.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:27.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:27.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_29 2024-04-23T21:46:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:46:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:46:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:46:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:46:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:46:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:46:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:46:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:46:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:46:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:27.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:46:27.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:27.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:46:27.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:27.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:27.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:27.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:27.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:27.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:27.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:27.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:27.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_29 2024-04-23T21:46:27.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:46:27.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:27.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:27.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:46:27.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:46:27.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:27.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:27.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:27.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:46:27.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:27.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:27.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:27.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:46:27.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:27.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:46:27.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:46:27.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:27.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:27.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:46:27.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:46:27.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:27.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:27.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:27.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:27.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:27.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:27.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:27.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:46:27.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:27.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:46:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:28.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:28.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_29 2024-04-23T21:46:28.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:28.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:28.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:28.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_29 '' 2024-04-23T21:46:28.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:28.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:28.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:28.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:46:28.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_29.status 2024-04-23T21:46:28.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_29.status 2024-04-23T21:46:28.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:28.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:28.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_29 2024-04-23T21:46:28.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:28.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:46:28.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:46:28.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:28.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8306], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:28.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=8306], [object_number=5, tag_tid=1, entry_tid=8305], [object_number=4, tag_tid=1, entry_tid=8304], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:28.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:28.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:46:28.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_29 2024-04-23T21:46:28.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:28.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:28.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:28.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_29 '..*' 2024-04-23T21:46:28.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:28.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:28.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:28.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:46:28.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_29.status 2024-04-23T21:46:28.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_29.status 2024-04-23T21:46:28.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:28.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:28.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_29 2024-04-23T21:46:28.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:28.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:46:28.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:46:28.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:28.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8306], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:28.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=8306], [object_number=5, tag_tid=1, entry_tid=8305], [object_number=4, tag_tid=1, entry_tid=8304], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:28.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:28.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:28.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:28.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=8306' '!=' 'object_number=6, tag_tid=1, entry_tid=8306' 2024-04-23T21:46:28.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:46:28.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:46:30.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_29 2024-04-23T21:46:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:46:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:46:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_29 ']' 2024-04-23T21:46:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_29' 2024-04-23T21:46:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_29 2024-04-23T21:46:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:46:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:46:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:46:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:46:30.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:30.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:46:30.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:30.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_29 2024-04-23T21:46:30.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:46:30.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:30.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:30.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:30.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:46:30.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:30.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:30.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:30.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:46:30.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:30.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:30.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:30.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:30.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:30.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:30.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:30.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:30.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:30.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2024-04-23T21:46:30.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:30.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_29 2024-04-23T21:46:30.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:46:30.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:30.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:30.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:30.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:46:30.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:30.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:30.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:30.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:46:30.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:30.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:30.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:30.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:46:30.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:30.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:30.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:46:30.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:30.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:30.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:30.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:30.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:30.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:30.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:30.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:30.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:30.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:30.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:30.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:30.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:30.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:30.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:30.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:30.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:30.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:30.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:30.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:30.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:30.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:30.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_29 2024-04-23T21:46:30.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:46:30.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:30.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:30.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:46:30.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:46:30.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:30.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:30.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:30.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:46:30.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:30.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:30.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:30.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:46:30.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:30.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:46:30.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:46:30.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:30.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:30.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:46:30.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:46:30.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:30.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:30.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:30.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:30.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:30.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:30.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:30.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:46:30.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:30.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:46:30.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:30.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:30.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:30.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:30.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2024-04-23T21:46:30.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:46:30.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:30.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:30.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_29 2024-04-23T21:46:30.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:46:30.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:46:30.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:46:30.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:46:30.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:46:30.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:46:30.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:46:30.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:46:30.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:46:30.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:30.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:46:30.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:30.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:46:30.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:30.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:46:30.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:46:30.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:30.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:30.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:46:30.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:46:30.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:46:30.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:30.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:30.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:30.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:30.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:46:30.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:46:30.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:46:30.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:30.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:46:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_29 2024-04-23T21:46:30.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:30.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_29 2024-04-23T21:46:30.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:30.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:30.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:30.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_29 '' 2024-04-23T21:46:30.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:30.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:30.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:30.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:46:30.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_29.status 2024-04-23T21:46:30.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_29.status 2024-04-23T21:46:30.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:30.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:30.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_29 2024-04-23T21:46:30.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:30.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:46:30.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:46:30.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:30.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8306], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:30.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8306], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:30.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:30.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:46:30.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_29 2024-04-23T21:46:30.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:30.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:30.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:30.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_29 '..*' 2024-04-23T21:46:30.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:46:30.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:46:30.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:46:30.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:46:30.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_29.status 2024-04-23T21:46:30.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_29.status 2024-04-23T21:46:30.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:46:30.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:30.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_29 2024-04-23T21:46:30.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:46:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:46:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:46:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8306], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:30.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8306], [object_number=7, tag_tid=1, entry_tid=8303]]], state=connected] 2024-04-23T21:46:30.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.status 2024-04-23T21:46:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:46:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:46:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_29 snap 2024-04-23T21:46:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:46:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:46:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:46:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:46:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:46:31.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_29@snap 2024-04-23T21:46:31.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_29': 2024-04-23T21:46:31.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:46:31.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:46:31.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:46:31.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10b5eac28ac6 2024-04-23T21:46:31.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10b5eac28ac6 2024-04-23T21:46:31.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:46:31.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:46:31.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:46:31.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:46:31.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:36:31 2024 2024-04-23T21:46:31.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:36:31 2024 2024-04-23T21:46:31.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:37:32 2024 2024-04-23T21:46:31.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:46:31.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10b5eac28ac6 2024-04-23T21:46:31.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:46:31.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:46:31.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 8053b610-1bd5-4698-aaa1-3b0ce0f05233 2024-04-23T21:46:31.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:46:31.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:31.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_29 snap 2024-04-23T21:46:31.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:31.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:46:31.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:46:31.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:46:31.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.export 2024-04-23T21:46:31.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_29.export 2024-04-23T21:46:31.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_29.export 2024-04-23T21:46:31.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_29@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.export 2024-04-23T21:46:31.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:46:31.883 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_29@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_29.export 2024-04-23T21:46:31.900 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8414253884547326) > weight (0.25) 2024-04-23T21:46:31.900 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4500963512497542) > weight (0.25) 2024-04-23T21:46:31.901 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2024-04-23T21:46:31.901 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2024-04-23T21:46:31.901 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2024-04-23T21:46:31.902 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi150.stderr:2024-04-23T21:46:31.901+0000 7f8fb04b0640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 57384) UID: 1000 2024-04-23T21:46:32.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:46:32.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_29.export 2024-04-23T21:46:32.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_29.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_29.export 2024-04-23T21:46:32.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:46:32.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:46:32.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_30 2024-04-23T21:46:32.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_30 snap 2024-04-23T21:46:32.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:46:32.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:32.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:46:32.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:46:32.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_30@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:46:33.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_30 2024-04-23T21:46:33.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:33.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:33.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:46:33.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_30 started 2024-04-23T21:46:33.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:33.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:33.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:46:33.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:46:33.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:46:33.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:33.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:46:34.295 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2024-04-23T21:46:34.295 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:46:34.296 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2024-04-23T21:46:34.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2024-04-23T21:46:34.296 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2024-04-23T21:46:34.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2024-04-23T21:46:34.299 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2024-04-23T21:46:34.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_30 started 2024-04-23T21:46:34.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:46:34.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:46:34.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:46:34.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:46:34.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:46:34.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:46:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_30 2024-04-23T21:46:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:46:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:46:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:46:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:46:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:46:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:34.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:46:34.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:34.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:46:34.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:34.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_30 2024-04-23T21:46:34.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:46:34.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:34.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:34.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:34.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:34.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:34.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:34.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:34.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:34.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:34.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:34.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:34.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:34.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:34.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:34.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:46:34.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:34.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:46:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:46:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:46:34.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:35.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:35.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:35.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:35.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_30 2024-04-23T21:46:35.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:35.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:35.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:35.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:35.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_30 2024-04-23T21:46:35.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:46:35.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:35.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:35.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:35.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:35.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:35.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:35.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:35.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:35.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:35.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:35.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:35.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:35.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:35.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:35.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:46:35.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:35.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:35.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:35.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:46:35.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:35.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:35.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:35.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:35.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:35.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:35.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:35.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:46:35.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:46:35.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:35.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:35.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:35.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:35.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_30 2024-04-23T21:46:35.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:35.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:35.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:35.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:35.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_30 2024-04-23T21:46:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:46:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:46:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:46:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:46:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:35.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:46:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:46:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:46:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:46:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:46:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:46:35.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:46:35.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:35.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:35.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:35.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_30 2024-04-23T21:46:35.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:35.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:35.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:35.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:35.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_30 2024-04-23T21:46:35.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:46:35.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:35.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:35.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:46:35.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:46:35.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:35.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:35.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:35.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:35.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:35.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:35.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:35.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:46:35.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:46:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:46:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:46:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:46:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:46:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:46:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:46:35.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:35.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:35.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:35.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_30 2024-04-23T21:46:35.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:35.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:35.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:35.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:35.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 1 2024-04-23T21:46:36.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:36.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:46:36.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:36.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_30 2024-04-23T21:46:36.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:46:36.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:36.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:36.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:46:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:36.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:36.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:46:36.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:36.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:36.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:36.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:36.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:36.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:36.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:36.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:46:36.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:46:36.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:36.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:36.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:36.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:36.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_30 2024-04-23T21:46:36.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:36.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:36.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:36.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:36.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_30 2024-04-23T21:46:36.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:46:36.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:36.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:36.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:46:36.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:36.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:36.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:36.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:46:36.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:36.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:36.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:36.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:36.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:36.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:36.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:36.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:46:36.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:46:36.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:36.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:36.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:36.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:36.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_30 2024-04-23T21:46:36.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:36.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:36.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_30 2024-04-23T21:46:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:46:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:46:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:46:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:36.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:46:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:46:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:46:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:46:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:46:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:46:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:46:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:46:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_30 2024-04-23T21:46:36.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:36.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:36.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:36.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:36.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_30 2024-04-23T21:46:36.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:46:36.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:36.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:36.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:46:36.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:46:36.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:36.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:36.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:36.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:36.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:36.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:36.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:36.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:46:36.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:36.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:46:36.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:46:36.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:36.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:36.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:46:36.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:46:36.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:36.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:36.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:36.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:36.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:36.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:36.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:36.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:46:36.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:46:36.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:46:36.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:36.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:36.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:36.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_30 2024-04-23T21:46:36.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:36.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:36.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:36.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:36.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 2 2024-04-23T21:46:38.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:38.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:46:38.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:38.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_30 2024-04-23T21:46:38.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:46:38.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:38.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:38.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:46:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:38.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:46:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:46:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:46:38.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:38.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:38.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:38.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:38.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_30 2024-04-23T21:46:38.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:38.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:38.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:38.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:38.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_30 2024-04-23T21:46:38.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:46:38.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:38.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:38.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:38.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:38.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:38.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:38.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:38.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:38.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:38.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:38.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:38.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:38.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:38.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:38.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:46:38.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:38.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:38.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:38.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:46:38.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:38.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:38.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:38.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:38.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:38.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:38.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:38.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:46:38.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:46:38.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:38.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:38.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:38.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:38.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_30 2024-04-23T21:46:39.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:39.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:39.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:39.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:39.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_30 2024-04-23T21:46:39.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:46:39.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:39.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:39.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:46:39.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:46:39.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:39.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:39.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:39.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:39.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:39.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:39.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:39.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:46:39.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:39.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:46:39.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:46:39.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:39.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:39.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:46:39.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:46:39.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:39.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:39.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:39.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:39.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:39.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:39.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:39.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:46:39.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:46:39.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:46:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_30 2024-04-23T21:46:39.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:39.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:39.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:39.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:39.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_30 2024-04-23T21:46:39.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:46:39.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:39.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:39.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:46:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:46:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:46:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:46:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:46:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:39.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:46:39.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:46:39.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:39.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:39.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:39.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:39.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:39.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:39.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:39.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:46:39.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:46:39.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:46:39.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:39.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:39.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:39.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_30 2024-04-23T21:46:39.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:39.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:39.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:39.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:39.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 4 2024-04-23T21:46:39.496 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:46:39.496+0000 7f83e7b51640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 1 2024-04-23T21:46:39.497 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:46:39.496+0000 7f83ee35e640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 2 2024-04-23T21:46:40.902 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2024-04-23T21:46:40.902 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:46:40.902 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2024-04-23T21:46:40.902 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2024-04-23T21:46:40.902 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2024-04-23T21:46:40.906 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2024-04-23T21:46:40.906 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2024-04-23T21:46:43.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:43.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:46:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_30 2024-04-23T21:46:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:46:43.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:43.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:46:43.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:43.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:43.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:43.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:46:43.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:43.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:43.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:43.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:43.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:43.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:43.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:43.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:46:43.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:46:43.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:43.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:43.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:43.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:43.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_30 2024-04-23T21:46:43.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:43.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:43.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:43.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:43.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_30 2024-04-23T21:46:43.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:46:43.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:43.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:43.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:43.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:43.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:43.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:43.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:43.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:43.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:43.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:43.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:43.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:43.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:43.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:43.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:46:43.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:43.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:43.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:43.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:46:43.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:43.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:43.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:43.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:43.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:43.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:43.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:43.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:46:43.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:46:43.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:43.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:43.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:43.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:43.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_30 2024-04-23T21:46:43.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:43.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:43.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:43.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:43.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_30 2024-04-23T21:46:43.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:46:43.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:43.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:43.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:46:43.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:46:43.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:43.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:43.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:43.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:43.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:43.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:43.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:46:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:46:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:46:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:46:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:46:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:43.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:43.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:46:43.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:46:43.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:46:43.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:43.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:43.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:43.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_30 2024-04-23T21:46:43.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:43.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:43.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_30 2024-04-23T21:46:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:46:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:46:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:46:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:43.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:46:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:46:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:46:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:46:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:46:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:43.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:43.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:43.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:43.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:43.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:43.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:43.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:46:43.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:46:43.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:46:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:43.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_30 2024-04-23T21:46:43.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:43.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:43.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:43.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:43.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 8 2024-04-23T21:46:51.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:46:51.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:46:51.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:51.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_30 2024-04-23T21:46:51.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:46:51.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:51.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:51.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:46:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:46:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:46:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:46:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:46:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:46:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:51.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:46:51.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:46:51.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:51.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:51.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:51.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:46:51.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:46:51.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:51.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:51.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:46:51.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:46:51.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:46:51.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:51.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:51.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:46:51.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_30 2024-04-23T21:46:51.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:51.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:51.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:51.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:51.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_30 2024-04-23T21:46:51.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:46:51.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:51.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:51.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:46:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:46:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:46:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:46:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:46:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:46:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:46:51.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:46:51.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:51.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:51.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:51.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:46:51.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:46:51.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:51.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:51.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:46:51.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:46:51.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:46:51.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:51.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:51.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:46:51.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_30 2024-04-23T21:46:51.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:51.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:51.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:51.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:51.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_30 2024-04-23T21:46:51.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:46:51.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:51.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:51.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:46:51.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:46:51.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:51.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:51.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:51.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:46:51.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:51.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:51.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:51.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:46:51.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:51.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:46:51.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:46:51.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:51.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:51.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:46:51.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:46:51.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:51.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:51.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:51.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:46:51.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:46:51.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:51.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:51.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:46:51.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:46:51.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:46:51.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:51.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:51.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:46:51.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_30 2024-04-23T21:46:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:51.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:51.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:46:51.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_30 2024-04-23T21:46:52.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:46:52.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:46:52.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:46:52.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:46:52.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:46:52.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:46:52.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:46:52.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:46:52.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:46:52.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:52.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:46:52.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:52.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:46:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:46:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:46:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:46:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:46:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:46:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:46:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:46:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:46:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:46:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:46:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:46:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:46:52.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:46:52.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:46:52.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:46:52.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:46:52.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:52.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:52.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:46:52.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_30 2024-04-23T21:46:52.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:46:52.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:46:52.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:46:52.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:46:52.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 8 2024-04-23T21:47:00.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:00.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:47:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_30 2024-04-23T21:47:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:47:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:00.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:47:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:47:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:47:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:47:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:47:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:00.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:47:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:47:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:47:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:47:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:47:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:47:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:47:00.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:00.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:00.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:00.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:00.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_30 2024-04-23T21:47:00.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:47:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:47:00.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:00.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:00.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_30 2024-04-23T21:47:00.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:47:00.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:00.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:00.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:00.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:00.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:47:00.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:00.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:00.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:00.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:00.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:47:00.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:00.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:00.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:00.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:00.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:47:00.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:47:00.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:47:00.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:00.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:47:00.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:47:00.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:47:00.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:00.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:00.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:00.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:47:00.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:47:00.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:47:00.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:47:00.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:00.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:00.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:00.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:00.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_30 2024-04-23T21:47:00.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:47:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:47:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:47:00.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:47:00.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:47:00.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:47:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:47:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_30 2024-04-23T21:47:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:47:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:47:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:47:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:47:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_30 2024-04-23T21:47:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:47:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:47:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:47:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:47:00.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:47:00.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:47:00.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:00.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:47:00.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_30 2024-04-23T21:47:00.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:00.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:00.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:47:00.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:00.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_30 ']' 2024-04-23T21:47:00.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_30' 2024-04-23T21:47:00.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_30 2024-04-23T21:47:00.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:00.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:00.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:00.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:00.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:00.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:00.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:00.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:00.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_30 2024-04-23T21:47:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:00.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:00.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:00.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:00.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:00.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:00.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:00.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:00.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:00.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:00.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:00.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:00.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:00.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:00.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:00.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:00.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:00.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:00.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_30 2024-04-23T21:47:00.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2024-04-23T21:47:00.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:00.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_30 2024-04-23T21:47:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:00.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:00.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:00.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:00.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:00.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:00.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:00.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_30 2024-04-23T21:47:00.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:00.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_30 2024-04-23T21:47:00.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:00.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:00.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_30 2024-04-23T21:47:00.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_30 '' 2024-04-23T21:47:00.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:00.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:00.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_30 2024-04-23T21:47:00.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_30.status 2024-04-23T21:47:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_30.status 2024-04-23T21:47:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_30.status 2024-04-23T21:47:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_30 2024-04-23T21:47:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_30.status 2024-04-23T21:47:00.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:00.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:00.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:00.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=9207], [object_number=10, tag_tid=1, entry_tid=9206]]], state=connected] 2024-04-23T21:47:00.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=9207], [object_number=10, tag_tid=1, entry_tid=9206]]], state=connected] 2024-04-23T21:47:00.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_30.status 2024-04-23T21:47:00.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:00.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_30 2024-04-23T21:47:00.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:00.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:00.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_30 2024-04-23T21:47:00.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_30 '..*' 2024-04-23T21:47:00.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:00.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:00.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_30 2024-04-23T21:47:00.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:00.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_30.status 2024-04-23T21:47:00.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_30.status 2024-04-23T21:47:00.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:00.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_30.status 2024-04-23T21:47:00.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_30 2024-04-23T21:47:00.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_30.status 2024-04-23T21:47:01.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:01.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:01.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:01.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=9207], [object_number=10, tag_tid=1, entry_tid=9206]]], state=connected] 2024-04-23T21:47:01.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=9207], [object_number=10, tag_tid=1, entry_tid=9206]]], state=connected] 2024-04-23T21:47:01.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_30.status 2024-04-23T21:47:01.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:01.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:01.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:01.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_30 snap 2024-04-23T21:47:01.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:01.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:01.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:47:01.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:47:01.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:47:01.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:47:01.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:47:02.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_30@snap 2024-04-23T21:47:02.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_30': 2024-04-23T21:47:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:47:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:47:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:47:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 116ee4f4f0e6 2024-04-23T21:47:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.116ee4f4f0e6 2024-04-23T21:47:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:47:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:47:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:47:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:47:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:36:40 2024 2024-04-23T21:47:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:36:40 2024 2024-04-23T21:47:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:37:41 2024 2024-04-23T21:47:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:47:02.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 116ee4f4f0e6 2024-04-23T21:47:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:47:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:47:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: a29e1728-a148-47d2-b738-171c51d259e1 2024-04-23T21:47:02.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:47:02.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_30 snap 2024-04-23T21:47:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:47:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:47:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:47:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_30.export 2024-04-23T21:47:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_30.export 2024-04-23T21:47:02.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_30.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_30.export 2024-04-23T21:47:02.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_30@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_30.export 2024-04-23T21:47:02.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:47:02.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_30@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_30.export 2024-04-23T21:47:02.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:47:02.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_30.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_30.export 2024-04-23T21:47:02.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_30.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_30.export 2024-04-23T21:47:02.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:02.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:47:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_31 2024-04-23T21:47:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_31 snap 2024-04-23T21:47:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:47:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:47:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:47:02.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_31@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:47:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_31 2024-04-23T21:47:05.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:05.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:05.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:47:05.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_31 started 2024-04-23T21:47:05.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:05.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:05.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:47:05.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:47:05.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:47:05.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:05.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:47:06.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_31 started 2024-04-23T21:47:06.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:47:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:47:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:47:06.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:47:06.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_31 2024-04-23T21:47:06.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:47:06.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:47:06.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:06.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:06.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:47:06.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:47:06.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:06.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:47:06.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:06.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:47:06.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:06.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_31 2024-04-23T21:47:06.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:47:06.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:06.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:06.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:06.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:06.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:47:06.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:06.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:06.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:06.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:06.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:47:06.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:06.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:06.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:06.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:06.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:47:06.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:47:06.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:47:06.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:06.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:47:06.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:47:06.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:47:06.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:06.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:06.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:06.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:47:06.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:47:06.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:47:06.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:47:06.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:06.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:06.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:06.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:06.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_31 2024-04-23T21:47:06.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:06.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:47:06.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:06.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:06.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_31 2024-04-23T21:47:06.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:47:06.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:47:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:47:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:47:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:47:06.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:47:06.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:06.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:47:06.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:47:06.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:47:06.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:06.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:06.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:06.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:47:06.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:47:06.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:47:06.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:47:06.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:06.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:06.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:06.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:06.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_31 2024-04-23T21:47:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:47:06.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:06.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:06.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_31 2024-04-23T21:47:06.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:47:06.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:06.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:06.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:47:06.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:47:06.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:47:06.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:06.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:06.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:47:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:47:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:47:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:47:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:47:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:47:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:47:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:47:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:47:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:47:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:06.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:06.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:47:06.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:47:06.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:47:06.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:47:06.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:47:06.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:06.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:06.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:06.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_31 2024-04-23T21:47:06.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:06.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:47:06.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:06.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:06.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_31 2024-04-23T21:47:06.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:47:06.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:06.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:06.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:47:06.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:47:06.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:47:06.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:06.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:06.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:47:06.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:47:06.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:47:06.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:06.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:47:06.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:47:06.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:47:06.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:47:06.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:47:06.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:47:06.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:47:06.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:47:06.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:47:06.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:47:06.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:06.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:47:06.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:47:06.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:47:06.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:47:06.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:47:06.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:47:06.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:47:06.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:06.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:06.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:06.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_31 2024-04-23T21:47:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:47:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:47:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:47:06.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:47:06.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:47:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:47:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:47:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_31 2024-04-23T21:47:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:47:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:47:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:47:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:47:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_31 2024-04-23T21:47:06.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:47:06.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:47:06.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:06.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:47:06.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:47:06.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:47:06.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:47:06.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:06.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:47:07.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_31 2024-04-23T21:47:07.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:07.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:07.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:47:07.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:07.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_31 ']' 2024-04-23T21:47:07.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_31' 2024-04-23T21:47:07.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_31 2024-04-23T21:47:07.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:07.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:07.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:07.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:07.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:07.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:07.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:07.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:07.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:07.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:07.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_31 2024-04-23T21:47:07.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:07.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:07.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:07.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:07.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:07.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:07.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:07.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:07.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:07.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:07.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:07.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:07.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:07.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:07.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:07.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:07.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:07.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:07.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:07.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:07.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:07.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:07.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:07.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:07.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:07.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:07.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:07.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:07.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:07.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:07.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:07.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:07.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:07.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:07.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:07.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:07.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_31 2024-04-23T21:47:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:07.118 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:07.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:07.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:07.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:07.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:07.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:07.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:07.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:07.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:07.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:07.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:07.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:07.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:07.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_31 2024-04-23T21:47:07.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:47:07.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:07.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:07.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:47:07.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:47:07.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:07.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:07.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:07.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:47:07.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:07.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:07.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:07.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:47:07.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:07.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:47:07.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:47:07.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:07.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:07.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:47:07.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:47:07.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:07.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:07.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:07.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:07.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:07.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:07.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:07.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:47:07.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:07.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:47:07.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:07.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:07.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:07.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:07.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:07.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:07.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:07.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:07.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_31 2024-04-23T21:47:07.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:47:07.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:07.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:07.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:47:07.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:47:07.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:07.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:07.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:07.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:47:07.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:47:07.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:07.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:07.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:47:07.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:47:07.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:47:07.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:47:07.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:07.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:07.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:47:07.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:47:07.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:07.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:07.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:07.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:47:07.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:47:07.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:07.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:07.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:47:07.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:47:07.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:47:07.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:07.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:07.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:07.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_31 2024-04-23T21:47:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_31 '' 2024-04-23T21:47:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_31.status 2024-04-23T21:47:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_31.status 2024-04-23T21:47:07.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:07.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:07.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_31 2024-04-23T21:47:07.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:07.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:07.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:07.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:07.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:07.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=10237], [object_number=8, tag_tid=1, entry_tid=10236], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:07.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:07.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:07.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_31 2024-04-23T21:47:07.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:07.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:07.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:07.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_31 '..*' 2024-04-23T21:47:07.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:07.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:07.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:07.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:07.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_31.status 2024-04-23T21:47:07.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_31.status 2024-04-23T21:47:07.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:07.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:07.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_31 2024-04-23T21:47:07.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:07.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:07.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:07.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:07.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=10237], [object_number=8, tag_tid=1, entry_tid=10236], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:07.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:07.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:07.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:07.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=10237' '!=' '' 2024-04-23T21:47:07.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:47:07.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=10237' = '' 2024-04-23T21:47:07.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:07.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:07.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:47:07.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:47:07.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:47:07.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:07.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:47:07.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:47:07.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:47:07.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:07.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:47:07.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:47:07.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:47:07.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:07.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:47:07.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:47:07.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=10237 2024-04-23T21:47:07.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 10237 2024-04-23T21:47:07.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:47:07.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:07.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:47:07.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_31 2024-04-23T21:47:07.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:07.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:07.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:47:07.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:07.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_31 ']' 2024-04-23T21:47:07.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_31' 2024-04-23T21:47:07.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_31 2024-04-23T21:47:07.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:07.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:07.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:07.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:07.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:07.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:07.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:07.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:07.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:07.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:07.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_31 2024-04-23T21:47:07.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:07.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:07.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:07.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:07.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:07.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:07.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:07.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:07.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:07.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:07.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:07.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:07.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:07.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:07.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:07.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:07.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:07.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:07.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:07.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:07.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:07.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:07.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:07.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:07.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:07.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:07.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:07.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:07.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_31 2024-04-23T21:47:07.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:07.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:07.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:07.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:07.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:07.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:07.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:07.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:07.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:07.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:07.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:07.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:07.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:07.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:07.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:07.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:07.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:07.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:07.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:07.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:07.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:07.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_31 2024-04-23T21:47:07.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:47:07.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:07.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:07.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:47:07.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:47:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:47:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:47:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:47:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:47:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:47:07.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:47:07.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:07.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:07.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:07.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:07.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:07.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:07.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:07.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:47:07.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:07.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:47:07.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:07.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:07.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:07.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:08.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:08.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:08.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:08.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:08.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_31 2024-04-23T21:47:08.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:47:08.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:08.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:08.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:47:08.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:47:08.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:08.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:08.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:08.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:47:08.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:47:08.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:08.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:08.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:47:08.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:47:08.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:47:08.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:47:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:47:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:47:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:47:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:47:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:47:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:47:08.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:47:08.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:08.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:08.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:08.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:08.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_31 2024-04-23T21:47:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_31 '' 2024-04-23T21:47:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_31.status 2024-04-23T21:47:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_31.status 2024-04-23T21:47:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:08.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:08.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_31 2024-04-23T21:47:08.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:08.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:08.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:08.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:08.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:08.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=10237], [object_number=8, tag_tid=1, entry_tid=10236], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:08.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_31 2024-04-23T21:47:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_31 '..*' 2024-04-23T21:47:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:08.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:08.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:08.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_31.status 2024-04-23T21:47:08.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_31.status 2024-04-23T21:47:08.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:08.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:08.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_31 2024-04-23T21:47:08.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:08.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:08.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:08.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:08.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:08.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=10237], [object_number=8, tag_tid=1, entry_tid=10236], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:08.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:08.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:08.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:08.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=10237' '!=' 'object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:08.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:08.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:47:08.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_31 2024-04-23T21:47:08.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:47:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_31 ']' 2024-04-23T21:47:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_31' 2024-04-23T21:47:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_31 2024-04-23T21:47:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:08.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:08.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_31 2024-04-23T21:47:08.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:08.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:08.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:08.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:08.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:08.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:08.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:08.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:08.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:08.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:08.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:08.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:08.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:08.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:08.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:08.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:08.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:08.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:08.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:08.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:08.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:08.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:08.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:08.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:08.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_31 2024-04-23T21:47:08.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:08.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:08.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:08.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:08.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:08.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:08.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:08.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:08.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:08.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:08.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:08.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:08.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:08.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:08.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:08.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:08.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:08.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:08.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:08.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:08.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:08.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:08.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:08.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:08.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:08.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:08.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:08.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:08.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:08.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:08.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:08.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:08.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:08.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:08.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:08.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:08.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:08.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:08.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_31 2024-04-23T21:47:08.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:47:08.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:08.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:08.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:47:08.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:47:08.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:08.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:08.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:08.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:47:08.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:08.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:08.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:08.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:47:08.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:08.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:47:08.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:47:08.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:08.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:08.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:47:08.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:47:08.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:08.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:08.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:08.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:08.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:08.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:08.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:08.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:47:08.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:08.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:47:08.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:08.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:08.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:08.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:08.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:08.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:08.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:08.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:08.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_31 2024-04-23T21:47:08.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:47:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:47:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:47:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:47:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:47:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:47:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:47:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:47:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:47:08.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:47:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:47:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:47:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:47:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:47:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:47:08.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:47:08.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:08.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:08.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:08.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:09.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_31 2024-04-23T21:47:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_31 '' 2024-04-23T21:47:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_31.status 2024-04-23T21:47:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_31.status 2024-04-23T21:47:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_31 2024-04-23T21:47:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:09.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:09.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:09.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:09.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:09.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=10237], [object_number=8, tag_tid=1, entry_tid=10236], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:09.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:09.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:09.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_31 2024-04-23T21:47:09.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:09.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:09.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:09.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_31 '..*' 2024-04-23T21:47:09.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:09.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:09.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:09.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:09.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_31.status 2024-04-23T21:47:09.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_31.status 2024-04-23T21:47:09.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:09.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:09.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_31 2024-04-23T21:47:09.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:09.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:09.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:09.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:09.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:09.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=10237], [object_number=8, tag_tid=1, entry_tid=10236], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:09.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:09.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:09.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:09.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=10237' '!=' 'object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:09.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:09.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:47:09.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_31 2024-04-23T21:47:09.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:09.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:09.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:47:09.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:09.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_31 ']' 2024-04-23T21:47:09.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_31' 2024-04-23T21:47:09.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_31 2024-04-23T21:47:09.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:09.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:09.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:09.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:09.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:09.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:09.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:09.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:09.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_31 2024-04-23T21:47:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:09.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:09.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:09.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:09.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:09.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:09.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:09.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:10.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:10.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:10.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:10.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:10.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_31 2024-04-23T21:47:10.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:10.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:10.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:10.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:10.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:10.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:10.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:10.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:10.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:10.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:10.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:10.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:10.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:10.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:10.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:10.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:10.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:10.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:10.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:10.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:10.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:10.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:10.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:10.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:10.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:10.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:10.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:10.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:10.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:10.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:10.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:10.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:10.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:10.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:10.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:10.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_31 2024-04-23T21:47:10.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:47:10.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:10.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:10.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:47:10.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:47:10.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:10.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:10.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:10.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:47:10.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:10.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:10.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:10.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:47:10.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:10.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:47:10.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:47:10.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:10.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:10.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:47:10.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:47:10.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:10.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:10.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:10.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:10.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:10.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:10.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:10.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:47:10.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:10.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:47:10.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:10.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:10.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:10.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:10.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:10.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:10.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:10.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:10.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_31 2024-04-23T21:47:10.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:47:10.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:10.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:10.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:47:10.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:47:10.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:10.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:10.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:10.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:47:10.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:47:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:47:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:47:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:47:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:47:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:47:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:47:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:47:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:47:10.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:10.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:10.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:47:10.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:47:10.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:47:10.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:10.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:10.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:10.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:10.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:10.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_31 2024-04-23T21:47:10.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:10.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:10.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:10.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_31 '' 2024-04-23T21:47:10.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:10.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:10.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:10.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:10.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_31.status 2024-04-23T21:47:10.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_31.status 2024-04-23T21:47:10.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:10.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:10.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_31 2024-04-23T21:47:10.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:10.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:10.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:10.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:10.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:10.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=10237], [object_number=8, tag_tid=1, entry_tid=10236], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:10.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:10.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:10.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_31 2024-04-23T21:47:10.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:10.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:10.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:10.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_31 '..*' 2024-04-23T21:47:10.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:10.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:10.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:10.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:10.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_31.status 2024-04-23T21:47:10.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_31.status 2024-04-23T21:47:10.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:10.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:10.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_31 2024-04-23T21:47:10.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:10.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:10.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:10.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:10.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:10.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=10237], [object_number=8, tag_tid=1, entry_tid=10236], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:10.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:10.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:10.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=10237' '!=' 'object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:10.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:10.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:47:12.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_31 2024-04-23T21:47:12.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:12.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:12.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:47:12.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:12.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_31 ']' 2024-04-23T21:47:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_31' 2024-04-23T21:47:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_31 2024-04-23T21:47:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:12.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:12.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:12.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:12.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:12.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_31 2024-04-23T21:47:12.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:12.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:12.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:12.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:12.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:12.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:12.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:12.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:12.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:12.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:12.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:12.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:12.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:12.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:12.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:12.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:12.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:12.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:12.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:12.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:12.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:12.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_31 2024-04-23T21:47:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:12.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:12.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:12.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:12.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:12.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:12.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:12.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:12.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:12.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:12.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:12.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:12.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_31 2024-04-23T21:47:12.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:47:12.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:12.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:12.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:47:12.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:47:12.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:12.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:12.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:12.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:47:12.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:12.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:12.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:12.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:47:12.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:12.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:47:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:47:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:47:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:47:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:47:12.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:12.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:47:12.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:12.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:12.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:12.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:12.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:12.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:12.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:12.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:12.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_31 2024-04-23T21:47:12.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:47:12.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:12.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:12.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:47:12.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:47:12.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:12.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:47:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:47:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:47:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:47:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:47:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:47:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:47:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:47:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:12.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:12.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:12.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:47:12.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:47:12.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:12.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:12.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:47:12.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:47:12.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:47:12.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:12.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:12.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:12.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_31 2024-04-23T21:47:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_31 '' 2024-04-23T21:47:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:12.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:12.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:12.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_31.status 2024-04-23T21:47:12.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_31.status 2024-04-23T21:47:12.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:12.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:12.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_31 2024-04-23T21:47:12.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:12.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:12.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:12.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:12.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:12.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=10237], [object_number=8, tag_tid=1, entry_tid=10236], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:12.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:12.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:12.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_31 2024-04-23T21:47:12.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:12.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:12.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:12.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_31 '..*' 2024-04-23T21:47:12.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:12.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:12.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:12.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:12.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_31.status 2024-04-23T21:47:12.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_31.status 2024-04-23T21:47:12.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:12.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:12.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_31 2024-04-23T21:47:12.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:12.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:12.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:12.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:12.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:12.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=1, entry_tid=10237], [object_number=8, tag_tid=1, entry_tid=10236], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:12.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:12.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:12.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:12.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=9, tag_tid=1, entry_tid=10237' '!=' 'object_number=9, tag_tid=1, entry_tid=10237' 2024-04-23T21:47:12.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:12.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:47:14.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_31 2024-04-23T21:47:14.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:14.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:14.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:47:14.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:14.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_31 ']' 2024-04-23T21:47:14.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_31' 2024-04-23T21:47:14.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_31 2024-04-23T21:47:14.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:14.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:14.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:14.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:14.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:14.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:14.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:14.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_31 2024-04-23T21:47:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:14.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:14.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:14.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:14.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:14.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:14.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:14.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:14.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:14.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:14.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:14.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:14.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:14.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:14.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:14.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:14.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:14.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:14.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:14.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:14.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_31 2024-04-23T21:47:14.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:14.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:14.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:14.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:14.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:14.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:14.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:14.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:14.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:14.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:14.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:14.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:14.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:14.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:14.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:14.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:14.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:14.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:14.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:14.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:14.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:14.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:14.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:14.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:14.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:14.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:14.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_31 2024-04-23T21:47:14.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:47:14.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:14.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:14.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:47:14.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:47:14.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:14.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:14.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:14.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:47:14.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:14.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:14.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:14.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:47:14.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:14.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:47:14.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:47:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:47:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:47:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:47:14.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:14.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:47:14.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:14.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:14.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:14.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:14.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2024-04-23T21:47:14.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:14.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:14.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:14.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_31 2024-04-23T21:47:14.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:47:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:47:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:47:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:47:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:47:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:47:14.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:47:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:47:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:47:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:47:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:47:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:47:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:47:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:14.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:47:14.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:47:14.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:47:14.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:14.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:14.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:47:14.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_31 2024-04-23T21:47:15.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:15.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_31 2024-04-23T21:47:15.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:15.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:15.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:15.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_31 '' 2024-04-23T21:47:15.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:15.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:15.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:15.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:15.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_31.status 2024-04-23T21:47:15.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_31.status 2024-04-23T21:47:15.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:15.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:15.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_31 2024-04-23T21:47:15.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:15.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:15.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:15.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:15.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:15.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:15.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:15.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:15.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_31 2024-04-23T21:47:15.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:15.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:15.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:15.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_31 '..*' 2024-04-23T21:47:15.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:15.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:15.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:47:15.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:15.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_31.status 2024-04-23T21:47:15.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_31.status 2024-04-23T21:47:15.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:15.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:15.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_31 2024-04-23T21:47:15.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:15.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:15.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:15.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:15.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:15.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10235], [object_number=10, tag_tid=1, entry_tid=10234]]], state=connected] 2024-04-23T21:47:15.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.status 2024-04-23T21:47:15.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:15.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:15.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:15.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_31 snap 2024-04-23T21:47:15.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:15.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:15.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:47:15.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:47:15.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:47:15.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:47:15.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:47:16.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_31@snap 2024-04-23T21:47:16.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_31': 2024-04-23T21:47:16.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:47:16.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:47:16.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:47:16.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 11b88c268086 2024-04-23T21:47:16.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.11b88c268086 2024-04-23T21:47:16.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:47:16.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:47:16.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:47:16.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:47:16.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:36:49 2024 2024-04-23T21:47:16.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:36:49 2024 2024-04-23T21:47:16.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:37:50 2024 2024-04-23T21:47:16.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:47:16.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 11b88c268086 2024-04-23T21:47:16.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:47:16.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:47:16.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: fbd02d69-b6c8-4f22-87f2-b319996d2191 2024-04-23T21:47:16.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:47:16.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:16.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_31 snap 2024-04-23T21:47:16.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:16.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:47:16.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:47:16.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:47:16.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.export 2024-04-23T21:47:16.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_31.export 2024-04-23T21:47:16.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_31.export 2024-04-23T21:47:16.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_31@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.export 2024-04-23T21:47:16.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:47:16.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_31@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_31.export 2024-04-23T21:47:16.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:47:16.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_31.export 2024-04-23T21:47:16.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_31.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_31.export 2024-04-23T21:47:16.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:16.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:47:16.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_32 2024-04-23T21:47:16.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_32 snap 2024-04-23T21:47:16.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:47:16.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:16.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:47:16.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:47:16.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_32@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:47:19.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_32 2024-04-23T21:47:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:47:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_32 started 2024-04-23T21:47:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:47:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:47:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:47:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:19.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:47:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_32 started 2024-04-23T21:47:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:47:20.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:47:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:47:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:47:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_32 2024-04-23T21:47:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:47:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:47:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:20.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:20.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:47:20.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:47:20.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:20.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:47:20.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:20.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:47:20.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:20.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_32 2024-04-23T21:47:20.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:47:20.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:20.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:20.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:20.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:20.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:47:20.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:20.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:20.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:20.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:20.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:47:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:47:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:47:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:47:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:47:20.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:47:20.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:47:20.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:20.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:20.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:20.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:47:20.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:47:20.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:47:20.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:47:20.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:20.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:20.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:20.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:20.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_32 2024-04-23T21:47:20.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2024-04-23T21:47:20.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:47:20.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:20.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:20.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_32 2024-04-23T21:47:20.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:47:20.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:20.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:20.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:20.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:20.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:47:20.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:20.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:20.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:20.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:20.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:47:20.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:20.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:20.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:20.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:20.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:47:20.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:47:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:47:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:47:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:47:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:47:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:47:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:47:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:47:20.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:47:20.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:20.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:20.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:20.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:20.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_32 2024-04-23T21:47:20.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2024-04-23T21:47:20.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:47:20.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:20.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:20.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_32 2024-04-23T21:47:20.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:47:20.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:20.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:20.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:47:20.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:47:20.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:47:20.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:20.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:20.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:20.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:20.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:47:20.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:20.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:47:20.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:20.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:47:20.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:47:20.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:47:20.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:47:20.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:47:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:47:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:47:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:47:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:47:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:47:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:47:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:47:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:47:20.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:20.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:20.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:20.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_32 2024-04-23T21:47:20.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:47:20.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:47:20.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:47:20.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:47:20.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:47:20.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:47:20.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:47:20.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:20.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_32 2024-04-23T21:47:20.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:47:20.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:47:20.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:20.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:47:20.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:47:20.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_32 2024-04-23T21:47:20.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:47:20.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:47:20.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:20.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:47:20.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:47:20.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:47:20.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:47:20.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:20.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:47:20.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_32 2024-04-23T21:47:20.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:20.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:20.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:47:20.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:20.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_32 ']' 2024-04-23T21:47:20.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_32' 2024-04-23T21:47:20.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_32 2024-04-23T21:47:20.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:20.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:20.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:20.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:20.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:20.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:20.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:20.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:20.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:20.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_32 2024-04-23T21:47:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:20.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:20.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:20.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:20.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:20.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:20.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:20.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:20.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:20.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:20.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:20.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:20.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:20.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:20.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:20.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:20.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:20.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:20.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:20.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:20.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:21.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2024-04-23T21:47:21.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:21.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:21.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_32 2024-04-23T21:47:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:21.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:21.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:21.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:21.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:21.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:21.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:21.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:21.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2024-04-23T21:47:21.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:21.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:21.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:21.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_32 2024-04-23T21:47:21.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:47:21.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:21.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:21.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:47:21.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:47:21.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:21.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:21.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:21.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:47:21.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:21.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:21.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:21.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:47:21.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:21.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:47:21.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:47:21.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:21.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:21.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:47:21.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:47:21.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:21.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:21.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:21.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:21.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:21.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:21.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:21.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:47:21.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:21.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:47:21.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:21.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:21.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:21.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:21.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:21.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_32 2024-04-23T21:47:21.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:21.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:21.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:21.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_32 '' 2024-04-23T21:47:21.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:21.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:21.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:21.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:21.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_32.status 2024-04-23T21:47:21.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:21.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_32.status 2024-04-23T21:47:21.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:21.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_32 2024-04-23T21:47:21.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=13306], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:21.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=13306], [object_number=13, tag_tid=1, entry_tid=13305], [object_number=12, tag_tid=1, entry_tid=13304], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:21.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:21.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:21.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_32 2024-04-23T21:47:21.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:21.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:21.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:21.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_32 '..*' 2024-04-23T21:47:21.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:21.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:21.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:21.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:21.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_32.status 2024-04-23T21:47:21.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_32.status 2024-04-23T21:47:21.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:21.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:21.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_32 2024-04-23T21:47:21.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:21.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:21.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:21.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:21.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=13306], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:21.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=13306], [object_number=13, tag_tid=1, entry_tid=13305], [object_number=12, tag_tid=1, entry_tid=13304], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:21.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=14, tag_tid=1, entry_tid=13306' '!=' '' 2024-04-23T21:47:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:47:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=14, tag_tid=1, entry_tid=13306' = '' 2024-04-23T21:47:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:47:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:47:21.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:47:21.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:21.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:47:21.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:47:21.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:47:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:47:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:47:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:47:21.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:21.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:47:21.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:47:21.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=13306 2024-04-23T21:47:21.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 13306 2024-04-23T21:47:21.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:47:21.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:21.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:47:21.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_32 2024-04-23T21:47:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:47:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_32 ']' 2024-04-23T21:47:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_32' 2024-04-23T21:47:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_32 2024-04-23T21:47:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:21.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:21.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:21.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:21.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:21.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:21.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_32 2024-04-23T21:47:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:21.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:21.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:21.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:21.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:21.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:21.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:21.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:21.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:21.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2024-04-23T21:47:21.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:21.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:21.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:21.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_32 2024-04-23T21:47:21.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:21.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:21.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:21.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:21.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:21.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:21.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:21.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:21.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:21.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:21.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:21.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:21.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:21.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:21.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:21.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:21.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:21.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:21.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:21.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:21.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2024-04-23T21:47:21.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:21.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_32 2024-04-23T21:47:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:47:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:47:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:47:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:47:21.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:21.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:21.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:47:21.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:21.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:47:21.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:21.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:21.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:21.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:21.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:21.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_32 2024-04-23T21:47:21.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:21.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:21.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:21.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_32 '' 2024-04-23T21:47:21.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:21.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:21.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:21.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:21.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_32.status 2024-04-23T21:47:21.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:21.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_32.status 2024-04-23T21:47:21.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:21.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_32 2024-04-23T21:47:21.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:21.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:21.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:21.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:21.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=13306], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:21.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=13306], [object_number=13, tag_tid=1, entry_tid=13305], [object_number=12, tag_tid=1, entry_tid=13304], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:21.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:21.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:21.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_32 2024-04-23T21:47:21.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:21.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:21.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:21.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_32 '..*' 2024-04-23T21:47:21.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:21.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:21.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:21.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:21.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_32.status 2024-04-23T21:47:21.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_32.status 2024-04-23T21:47:21.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:21.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:21.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_32 2024-04-23T21:47:21.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:21.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:21.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:21.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:21.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=13306], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:21.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=13306], [object_number=13, tag_tid=1, entry_tid=13305], [object_number=12, tag_tid=1, entry_tid=13304], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:21.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:21.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:21.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:21.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=14, tag_tid=1, entry_tid=13306' '!=' 'object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:21.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:21.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:47:22.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_32 2024-04-23T21:47:22.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:22.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:22.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:47:22.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:22.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_32 ']' 2024-04-23T21:47:22.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_32' 2024-04-23T21:47:22.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_32 2024-04-23T21:47:22.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:22.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:22.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:22.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:22.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:22.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:22.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:22.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:22.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:22.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:22.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_32 2024-04-23T21:47:22.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:22.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:22.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:22.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:22.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:22.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:22.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:22.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:22.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2024-04-23T21:47:22.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:22.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:22.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:22.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_32 2024-04-23T21:47:22.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:22.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:22.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:22.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:22.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:22.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:22.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:22.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:22.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2024-04-23T21:47:22.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:22.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:22.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:22.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_32 2024-04-23T21:47:22.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:47:22.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:22.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:22.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:47:22.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:47:22.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:22.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:22.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:22.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:47:22.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:22.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:22.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:22.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:47:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:47:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:47:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:47:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:47:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:22.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:47:22.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:22.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:47:22.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:22.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:22.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:22.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:22.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:22.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_32 2024-04-23T21:47:22.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:22.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:22.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:22.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_32 '' 2024-04-23T21:47:22.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:22.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:22.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:22.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:22.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_32.status 2024-04-23T21:47:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_32.status 2024-04-23T21:47:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_32 2024-04-23T21:47:22.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:22.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:22.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:22.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:22.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=13306], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:22.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=13306], [object_number=13, tag_tid=1, entry_tid=13305], [object_number=12, tag_tid=1, entry_tid=13304], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:22.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:22.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_32 2024-04-23T21:47:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:22.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_32 '..*' 2024-04-23T21:47:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_32.status 2024-04-23T21:47:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_32.status 2024-04-23T21:47:22.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:22.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:22.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_32 2024-04-23T21:47:22.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:22.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:22.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:22.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:22.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=13306], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:22.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=13306], [object_number=13, tag_tid=1, entry_tid=13305], [object_number=12, tag_tid=1, entry_tid=13304], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:22.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:22.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:22.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:22.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=14, tag_tid=1, entry_tid=13306' '!=' 'object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:22.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:22.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:47:23.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_32 2024-04-23T21:47:23.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:23.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:23.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:47:23.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:23.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_32 ']' 2024-04-23T21:47:23.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_32' 2024-04-23T21:47:23.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_32 2024-04-23T21:47:23.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:23.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:23.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:23.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:23.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:23.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:23.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:23.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_32 2024-04-23T21:47:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:23.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:23.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:23.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:23.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:23.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:23.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:23.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:23.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:23.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:23.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:23.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:23.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:23.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:23.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:23.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:23.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:23.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:23.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:23.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:23.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:23.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:23.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:23.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:23.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:23.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:23.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:23.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:23.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2024-04-23T21:47:23.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:23.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:23.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:23.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_32 2024-04-23T21:47:23.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:23.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:23.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:23.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:23.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:23.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:23.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:23.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:23.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:23.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:23.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:23.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:23.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:23.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:23.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:23.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:23.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:23.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:23.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:23.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:23.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:23.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:23.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2024-04-23T21:47:23.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:23.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:23.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:23.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_32 2024-04-23T21:47:23.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:47:23.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:23.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:23.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:47:23.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:47:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:47:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:47:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:47:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:47:23.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:47:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:47:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:47:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:23.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:47:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:23.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_32 2024-04-23T21:47:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_32 '' 2024-04-23T21:47:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:23.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_32.status 2024-04-23T21:47:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_32.status 2024-04-23T21:47:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_32 2024-04-23T21:47:23.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:23.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:23.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:23.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:23.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=13306], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:23.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=13306], [object_number=13, tag_tid=1, entry_tid=13305], [object_number=12, tag_tid=1, entry_tid=13304], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:23.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:23.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_32 2024-04-23T21:47:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_32 '..*' 2024-04-23T21:47:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_32.status 2024-04-23T21:47:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_32.status 2024-04-23T21:47:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_32 2024-04-23T21:47:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=13306], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=13306], [object_number=13, tag_tid=1, entry_tid=13305], [object_number=12, tag_tid=1, entry_tid=13304], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:23.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=14, tag_tid=1, entry_tid=13306' '!=' 'object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:47:25.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_32 2024-04-23T21:47:25.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:25.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:25.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:47:25.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:25.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_32 ']' 2024-04-23T21:47:25.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_32' 2024-04-23T21:47:25.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_32 2024-04-23T21:47:25.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:25.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:25.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:25.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:25.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:25.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:25.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:25.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_32 2024-04-23T21:47:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:25.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:25.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:25.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:25.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:25.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:25.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:25.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:25.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2024-04-23T21:47:25.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:25.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:25.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_32 2024-04-23T21:47:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:25.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:25.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:25.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:25.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:25.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:25.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:25.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2024-04-23T21:47:25.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:25.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:25.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:25.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_32 2024-04-23T21:47:25.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:47:25.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:25.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:25.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:47:25.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:47:25.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:25.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:25.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:25.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:47:25.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:25.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:25.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:25.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:47:25.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:25.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:47:25.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:47:25.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:25.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:25.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:47:25.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:47:25.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:25.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:25.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:25.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:25.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:25.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:25.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:25.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:47:25.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:25.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:47:25.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:25.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:25.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:25.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:25.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:25.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_32 2024-04-23T21:47:25.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:25.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:25.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:25.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_32 '' 2024-04-23T21:47:25.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:25.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:25.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:25.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:25.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_32.status 2024-04-23T21:47:25.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_32.status 2024-04-23T21:47:25.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:25.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:25.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_32 2024-04-23T21:47:25.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:25.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:25.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:25.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:25.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=13306], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:25.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=13306], [object_number=13, tag_tid=1, entry_tid=13305], [object_number=12, tag_tid=1, entry_tid=13304], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:25.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:25.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:25.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_32 2024-04-23T21:47:25.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:25.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:25.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:25.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_32 '..*' 2024-04-23T21:47:25.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:25.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:25.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:25.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:25.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_32.status 2024-04-23T21:47:25.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_32.status 2024-04-23T21:47:25.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:25.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:25.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_32 2024-04-23T21:47:25.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=13306], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:25.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=13306], [object_number=13, tag_tid=1, entry_tid=13305], [object_number=12, tag_tid=1, entry_tid=13304], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:25.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:25.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:25.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:25.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=14, tag_tid=1, entry_tid=13306' '!=' 'object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:25.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:25.980 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:47:27.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_32 2024-04-23T21:47:27.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:27.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:27.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:47:27.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:27.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_32 ']' 2024-04-23T21:47:27.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_32' 2024-04-23T21:47:27.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_32 2024-04-23T21:47:27.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:27.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:27.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:27.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:27.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:27.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:27.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:27.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:27.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:27.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:27.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_32 2024-04-23T21:47:27.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:27.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:27.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:27.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:27.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:27.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:27.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:27.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:27.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:27.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:27.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:27.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:27.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:27.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:27.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:28.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:28.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:28.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:28.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:28.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2024-04-23T21:47:28.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:28.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:28.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:28.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_32 2024-04-23T21:47:28.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:28.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:28.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:28.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:28.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:28.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:28.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:28.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:28.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:28.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:28.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:28.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:28.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:28.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:28.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:28.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:28.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:28.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:28.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:28.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:28.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:28.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:28.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:28.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:28.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:28.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:28.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:28.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:28.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:28.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:28.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:28.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:28.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:28.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:28.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2024-04-23T21:47:28.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:28.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:28.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:28.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_32 2024-04-23T21:47:28.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:47:28.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:28.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:28.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:47:28.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:47:28.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:28.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:28.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:28.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:47:28.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:28.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:28.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:28.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:47:28.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:28.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:47:28.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:47:28.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:28.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:28.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:47:28.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:47:28.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:28.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:28.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:28.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:28.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:28.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:28.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:28.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:47:28.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:28.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:47:28.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:28.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:28.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:28.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:28.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:28.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_32 2024-04-23T21:47:28.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:28.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:28.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:28.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_32 '' 2024-04-23T21:47:28.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:28.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:28.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:28.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:28.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_32.status 2024-04-23T21:47:28.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_32.status 2024-04-23T21:47:28.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:28.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:28.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_32 2024-04-23T21:47:28.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:28.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:28.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:28.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:28.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=13306], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:28.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=13306], [object_number=13, tag_tid=1, entry_tid=13305], [object_number=12, tag_tid=1, entry_tid=13304], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:28.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:28.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:28.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_32 2024-04-23T21:47:28.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:28.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:28.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:28.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_32 '..*' 2024-04-23T21:47:28.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:28.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:28.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:28.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:28.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_32.status 2024-04-23T21:47:28.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_32.status 2024-04-23T21:47:28.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:28.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:28.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_32 2024-04-23T21:47:28.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:28.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:28.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:28.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:28.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=13306], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:28.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=13306], [object_number=13, tag_tid=1, entry_tid=13305], [object_number=12, tag_tid=1, entry_tid=13304], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:28.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:28.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:28.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:28.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=14, tag_tid=1, entry_tid=13306' '!=' 'object_number=14, tag_tid=1, entry_tid=13306' 2024-04-23T21:47:28.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:28.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:47:30.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_32 2024-04-23T21:47:30.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:30.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:30.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:47:30.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:30.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_32 ']' 2024-04-23T21:47:30.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_32' 2024-04-23T21:47:30.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_32 2024-04-23T21:47:30.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:30.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:30.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:30.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:30.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:30.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:30.398 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_32 2024-04-23T21:47:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:30.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:30.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:30.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:30.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:30.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:30.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2024-04-23T21:47:30.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:30.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:30.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:30.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_32 2024-04-23T21:47:30.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:30.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:30.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:30.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:30.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:30.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:30.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:30.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:30.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:30.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:30.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:30.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:30.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:30.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:30.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:30.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:30.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:30.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:30.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:30.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:30.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:30.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:30.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:30.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:30.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:30.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:30.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:30.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:30.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:30.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:30.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:30.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:30.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:30.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2024-04-23T21:47:30.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:30.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:30.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:30.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_32 2024-04-23T21:47:30.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:47:30.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:30.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:30.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:47:30.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:47:30.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:30.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:30.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:47:30.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:47:30.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:30.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:30.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:47:30.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:47:30.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:47:30.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:30.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:30.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:47:30.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_32 2024-04-23T21:47:30.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:30.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_32 2024-04-23T21:47:30.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:30.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:30.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:30.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_32 '' 2024-04-23T21:47:30.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:30.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:30.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:30.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:30.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_32.status 2024-04-23T21:47:30.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_32.status 2024-04-23T21:47:30.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:30.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:30.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_32 2024-04-23T21:47:30.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:30.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:30.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:30.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:30.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=13306], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:30.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=13306], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:30.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:30.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:30.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_32 2024-04-23T21:47:30.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:30.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:30.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:30.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_32 '..*' 2024-04-23T21:47:30.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:30.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:30.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:47:30.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:30.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_32.status 2024-04-23T21:47:30.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_32.status 2024-04-23T21:47:30.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:30.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:30.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_32 2024-04-23T21:47:30.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=13306], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:30.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=13306], [object_number=15, tag_tid=1, entry_tid=13303]]], state=connected] 2024-04-23T21:47:30.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.status 2024-04-23T21:47:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_32 snap 2024-04-23T21:47:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:47:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:47:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:47:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:47:30.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:47:31.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_32@snap 2024-04-23T21:47:31.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_32': 2024-04-23T21:47:31.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:47:31.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:47:31.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:47:31.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 121daaddfd8d 2024-04-23T21:47:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.121daaddfd8d 2024-04-23T21:47:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:47:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:47:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:47:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:47:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:36:59 2024 2024-04-23T21:47:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:36:59 2024 2024-04-23T21:47:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:38:00 2024 2024-04-23T21:47:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:47:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 121daaddfd8d 2024-04-23T21:47:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:47:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:47:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 332a160e-6939-4ab3-b920-daf4089d188f 2024-04-23T21:47:31.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:47:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_32 snap 2024-04-23T21:47:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:47:31.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:47:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:47:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.export 2024-04-23T21:47:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_32.export 2024-04-23T21:47:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_32.export 2024-04-23T21:47:31.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_32@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.export 2024-04-23T21:47:32.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:47:32.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_32@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_32.export 2024-04-23T21:47:32.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:47:32.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_32.export 2024-04-23T21:47:32.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_32.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_32.export 2024-04-23T21:47:32.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:32.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:47:32.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_33 2024-04-23T21:47:32.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_33 snap 2024-04-23T21:47:32.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:47:32.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:32.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:47:32.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:47:32.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_33@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:47:34.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_33 2024-04-23T21:47:34.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:34.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:34.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:47:34.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_33 started 2024-04-23T21:47:34.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:34.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:34.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:47:34.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:47:34.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:47:34.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:34.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:47:35.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_33 started 2024-04-23T21:47:35.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:35.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:47:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:47:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:47:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:47:35.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_33 2024-04-23T21:47:35.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:47:35.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:47:35.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:35.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:35.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:47:35.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:47:35.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:35.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:47:35.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:35.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:47:35.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_33 2024-04-23T21:47:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:47:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:47:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:47:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:47:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:47:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:47:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:47:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:47:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:47:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:47:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:47:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:47:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:47:35.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:35.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:35.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:35.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:35.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_33 2024-04-23T21:47:35.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_33'] 2024-04-23T21:47:35.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:47:35.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:35.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:35.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_33 2024-04-23T21:47:35.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:47:35.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:35.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:35.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:35.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:35.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:47:35.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:35.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:35.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:35.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:35.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:47:35.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:35.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:35.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:35.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:35.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:47:35.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:47:35.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:47:35.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:35.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:47:35.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:47:35.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:47:35.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:35.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:35.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:35.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:47:35.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:47:35.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:47:35.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:47:35.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:35.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:35.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:35.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:35.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_33 2024-04-23T21:47:35.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:47:35.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:47:35.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:47:35.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:47:35.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:47:35.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:47:35.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:47:35.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:35.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_33 2024-04-23T21:47:35.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:47:35.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:47:35.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:35.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:47:35.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:47:35.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_33 2024-04-23T21:47:35.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:47:35.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:47:35.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:35.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:47:35.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:47:35.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:47:35.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:47:35.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:35.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:47:35.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_33 2024-04-23T21:47:35.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:35.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:35.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:47:35.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:35.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_33 ']' 2024-04-23T21:47:35.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_33' 2024-04-23T21:47:35.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_33 2024-04-23T21:47:35.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:35.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_33 2024-04-23T21:47:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:35.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:35.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:35.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:35.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:35.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_33 2024-04-23T21:47:35.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_33'] 2024-04-23T21:47:35.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:35.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:35.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:35.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_33 2024-04-23T21:47:35.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:35.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:35.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:35.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:35.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:35.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:35.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:35.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:35.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:35.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:35.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:35.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:35.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:35.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:35.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:35.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:35.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:35.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:35.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_33 2024-04-23T21:47:36.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:36.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_33 2024-04-23T21:47:36.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:36.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:36.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:36.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_33 '' 2024-04-23T21:47:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_33.status 2024-04-23T21:47:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_33.status 2024-04-23T21:47:36.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_33 2024-04-23T21:47:36.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:36.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:36.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:36.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:36.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038]]], state=connected] 2024-04-23T21:47:36.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038], [object_number=9, tag_tid=1, entry_tid=12037], [object_number=8, tag_tid=1, entry_tid=12036]]], state=connected] 2024-04-23T21:47:36.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:36.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:36.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_33 2024-04-23T21:47:36.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:36.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:36.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:36.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_33 '..*' 2024-04-23T21:47:36.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:36.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:36.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:36.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:36.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_33.status 2024-04-23T21:47:36.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_33.status 2024-04-23T21:47:36.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:36.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:36.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_33 2024-04-23T21:47:36.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:36.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:36.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:36.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:36.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038]]], state=connected] 2024-04-23T21:47:36.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038], [object_number=9, tag_tid=1, entry_tid=12037], [object_number=8, tag_tid=1, entry_tid=12036]]], state=connected] 2024-04-23T21:47:36.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:36.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=11, tag_tid=1, entry_tid=12039' 2024-04-23T21:47:36.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=11, tag_tid=1, entry_tid=12039' 2024-04-23T21:47:36.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=12039' '!=' '' 2024-04-23T21:47:36.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:47:36.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=12039' = '' 2024-04-23T21:47:36.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=11, tag_tid=1, entry_tid=12039' 2024-04-23T21:47:36.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:36.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:47:36.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:47:36.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:47:36.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=11, tag_tid=1, entry_tid=12039' 2024-04-23T21:47:36.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:47:36.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:47:36.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:47:36.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:36.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:47:36.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:47:36.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:47:36.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=11, tag_tid=1, entry_tid=12039' 2024-04-23T21:47:36.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:47:36.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:47:36.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=12039 2024-04-23T21:47:36.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 12039 2024-04-23T21:47:36.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:47:36.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:36.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:47:36.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_33 2024-04-23T21:47:36.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:36.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:47:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_33 ']' 2024-04-23T21:47:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_33' 2024-04-23T21:47:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_33 2024-04-23T21:47:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:36.358 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:36.359 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_33 2024-04-23T21:47:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:36.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:36.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:36.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:36.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:36.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:36.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:36.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_33 2024-04-23T21:47:36.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_33'] 2024-04-23T21:47:36.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:36.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:36.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:36.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_33 2024-04-23T21:47:36.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:36.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:36.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:36.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:36.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:36.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:36.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:36.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:36.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:36.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:36.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:36.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:36.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:36.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:36.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:36.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:36.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:36.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:36.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:36.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:36.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_33 2024-04-23T21:47:36.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:36.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_33 2024-04-23T21:47:36.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:36.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:36.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:36.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_33 '' 2024-04-23T21:47:36.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:36.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:36.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:36.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:36.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_33.status 2024-04-23T21:47:36.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_33.status 2024-04-23T21:47:36.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:36.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:36.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_33 2024-04-23T21:47:36.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:36.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038]]], state=connected] 2024-04-23T21:47:36.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038], [object_number=9, tag_tid=1, entry_tid=12037], [object_number=8, tag_tid=1, entry_tid=12036]]], state=connected] 2024-04-23T21:47:36.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:36.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_33 2024-04-23T21:47:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_33 '..*' 2024-04-23T21:47:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_33.status 2024-04-23T21:47:36.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_33.status 2024-04-23T21:47:36.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:36.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:36.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_33 2024-04-23T21:47:36.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:36.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:36.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:36.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:36.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038]]], state=connected] 2024-04-23T21:47:36.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038], [object_number=9, tag_tid=1, entry_tid=12037], [object_number=8, tag_tid=1, entry_tid=12036]]], state=connected] 2024-04-23T21:47:36.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:36.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=11, tag_tid=1, entry_tid=12039' 2024-04-23T21:47:36.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=11, tag_tid=1, entry_tid=12039' 2024-04-23T21:47:36.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=12039' '!=' 'object_number=11, tag_tid=1, entry_tid=12039' 2024-04-23T21:47:36.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:36.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:47:37.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_33 2024-04-23T21:47:37.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:37.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:37.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:47:37.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:37.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_33 ']' 2024-04-23T21:47:37.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_33' 2024-04-23T21:47:37.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_33 2024-04-23T21:47:37.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:37.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:37.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:37.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:37.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:37.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_33 2024-04-23T21:47:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:37.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:37.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:37.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:37.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:37.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:37.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:37.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:37.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:37.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:37.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:37.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:37.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:37.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:37.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:37.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:37.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:37.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:37.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:37.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:37.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:37.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:37.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:37.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:37.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:37.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_33 2024-04-23T21:47:37.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_33'] 2024-04-23T21:47:37.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:37.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:37.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:37.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_33 2024-04-23T21:47:37.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:37.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:37.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:37.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:37.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:37.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:37.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:37.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:37.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:37.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:37.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:37.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_33 2024-04-23T21:47:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_33 2024-04-23T21:47:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_33 '' 2024-04-23T21:47:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:37.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_33.status 2024-04-23T21:47:37.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_33.status 2024-04-23T21:47:37.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:37.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:37.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_33 2024-04-23T21:47:37.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:37.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:37.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:37.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:37.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038]]], state=connected] 2024-04-23T21:47:37.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038], [object_number=9, tag_tid=1, entry_tid=12037], [object_number=8, tag_tid=1, entry_tid=12036]]], state=connected] 2024-04-23T21:47:37.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:37.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:37.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_33 2024-04-23T21:47:37.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:37.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:37.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:37.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_33 '..*' 2024-04-23T21:47:37.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:37.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:37.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:37.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:37.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_33.status 2024-04-23T21:47:37.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_33.status 2024-04-23T21:47:37.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:37.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:37.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_33 2024-04-23T21:47:37.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:37.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:37.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:37.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:37.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038]]], state=connected] 2024-04-23T21:47:37.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038], [object_number=9, tag_tid=1, entry_tid=12037], [object_number=8, tag_tid=1, entry_tid=12036]]], state=connected] 2024-04-23T21:47:37.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:37.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=11, tag_tid=1, entry_tid=12039' 2024-04-23T21:47:37.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=11, tag_tid=1, entry_tid=12039' 2024-04-23T21:47:37.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=12039' '!=' 'object_number=11, tag_tid=1, entry_tid=12039' 2024-04-23T21:47:37.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:37.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:47:38.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_33 2024-04-23T21:47:38.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:38.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:38.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:47:38.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:38.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_33 ']' 2024-04-23T21:47:38.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_33' 2024-04-23T21:47:38.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_33 2024-04-23T21:47:38.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:38.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:38.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:38.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:38.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:38.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:38.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:38.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:38.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:38.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:38.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_33 2024-04-23T21:47:38.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:38.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:38.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:38.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:38.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:38.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:38.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:38.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:38.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:38.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:38.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:38.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:38.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:38.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:38.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:38.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:38.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:38.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:38.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:38.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_33 2024-04-23T21:47:38.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_33'] 2024-04-23T21:47:38.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:38.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:38.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:38.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_33 2024-04-23T21:47:38.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:38.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:38.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:38.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:38.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:38.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:38.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:38.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:38.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:38.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:38.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:38.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:38.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:38.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:38.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:38.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:38.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:38.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:38.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:38.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:38.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_33 2024-04-23T21:47:38.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:38.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_33 2024-04-23T21:47:38.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:38.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:38.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:38.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_33 '' 2024-04-23T21:47:38.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:38.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:38.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:38.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:38.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_33.status 2024-04-23T21:47:38.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_33.status 2024-04-23T21:47:38.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:38.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:38.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_33 2024-04-23T21:47:38.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:38.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:38.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:38.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:38.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038]]], state=connected] 2024-04-23T21:47:38.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038], [object_number=9, tag_tid=1, entry_tid=12037], [object_number=8, tag_tid=1, entry_tid=12036]]], state=connected] 2024-04-23T21:47:38.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:38.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:38.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_33 2024-04-23T21:47:38.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:38.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:38.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:38.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_33 '..*' 2024-04-23T21:47:38.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:38.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:38.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:38.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:38.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_33.status 2024-04-23T21:47:38.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_33.status 2024-04-23T21:47:38.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:38.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:38.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_33 2024-04-23T21:47:38.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038]]], state=connected] 2024-04-23T21:47:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038], [object_number=9, tag_tid=1, entry_tid=12037], [object_number=8, tag_tid=1, entry_tid=12036]]], state=connected] 2024-04-23T21:47:38.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:38.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=11, tag_tid=1, entry_tid=12039' 2024-04-23T21:47:38.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=11, tag_tid=1, entry_tid=12039' 2024-04-23T21:47:38.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=12039' '!=' 'object_number=11, tag_tid=1, entry_tid=12039' 2024-04-23T21:47:38.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:38.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:47:40.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_33 2024-04-23T21:47:40.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:40.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:40.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:47:40.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:40.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_33 ']' 2024-04-23T21:47:40.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_33' 2024-04-23T21:47:40.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_33 2024-04-23T21:47:40.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:40.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:40.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:40.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:40.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:40.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:40.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:40.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_33 2024-04-23T21:47:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:40.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:40.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:40.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:40.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:40.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:40.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:40.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:40.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:40.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:40.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:40.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:40.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:40.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_33 2024-04-23T21:47:40.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_33'] 2024-04-23T21:47:40.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:40.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:40.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:40.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_33 2024-04-23T21:47:40.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:40.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:40.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:40.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:40.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:40.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:40.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:40.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:40.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:40.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:40.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:40.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:40.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:40.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:40.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:40.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:40.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:40.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:40.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:40.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:40.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:40.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:40.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:40.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:40.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:40.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:40.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:40.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:40.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:40.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:40.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:40.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:40.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:40.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_33 2024-04-23T21:47:40.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:40.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_33 2024-04-23T21:47:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_33 '' 2024-04-23T21:47:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_33.status 2024-04-23T21:47:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_33.status 2024-04-23T21:47:40.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:40.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:40.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_33 2024-04-23T21:47:40.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:40.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:40.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:40.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:40.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038]]], state=connected] 2024-04-23T21:47:40.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038]]], state=connected] 2024-04-23T21:47:40.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:40.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:40.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_33 2024-04-23T21:47:40.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:40.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:40.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:40.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_33 '..*' 2024-04-23T21:47:40.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:40.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:40.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:47:40.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:40.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_33.status 2024-04-23T21:47:40.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_33.status 2024-04-23T21:47:40.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:40.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:40.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_33 2024-04-23T21:47:40.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:40.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:47:40.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:47:40.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:40.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038]]], state=connected] 2024-04-23T21:47:40.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=12039], [object_number=10, tag_tid=1, entry_tid=12038]]], state=connected] 2024-04-23T21:47:40.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.status 2024-04-23T21:47:40.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:40.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:47:40.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:40.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_33 snap 2024-04-23T21:47:40.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:40.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:40.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:47:40.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:47:40.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:47:40.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:47:40.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:47:41.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_33@snap 2024-04-23T21:47:41.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_33': 2024-04-23T21:47:41.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:47:41.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:47:41.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:47:41.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10b526536cb1 2024-04-23T21:47:41.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10b526536cb1 2024-04-23T21:47:41.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:47:41.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:47:41.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:47:41.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:47:41.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:37:11 2024 2024-04-23T21:47:41.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:37:11 2024 2024-04-23T21:47:41.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:39:16 2024 2024-04-23T21:47:41.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:47:41.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10b526536cb1 2024-04-23T21:47:41.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:47:41.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:47:41.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: f9c997ae-ac52-4b6f-a5c9-895a60fa5ef8 2024-04-23T21:47:41.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:47:41.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:41.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_33 snap 2024-04-23T21:47:41.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:41.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:47:41.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:47:41.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:47:41.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.export 2024-04-23T21:47:41.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_33.export 2024-04-23T21:47:41.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_33.export 2024-04-23T21:47:41.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_33@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.export 2024-04-23T21:47:41.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:47:41.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_33@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_33.export 2024-04-23T21:47:42.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:47:42.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_33.export 2024-04-23T21:47:42.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_33.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_33.export 2024-04-23T21:47:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:47:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_34 2024-04-23T21:47:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_34 snap 2024-04-23T21:47:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:47:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:47:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:47:42.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_34@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:47:42.299 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9357187207566634) > weight (0.25) 2024-04-23T21:47:42.300 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4740232287815179) > weight (0.25) 2024-04-23T21:47:42.300 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7776729341365154) > weight (0.25) 2024-04-23T21:47:42.300 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7624913792075174) > weight (0.25) 2024-04-23T21:47:42.300 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2024-04-23T21:47:44.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_34 2024-04-23T21:47:44.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:44.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:44.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:47:44.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_34 started 2024-04-23T21:47:44.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:44.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:44.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:47:44.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:47:44.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:47:44.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:44.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:47:45.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_34 started 2024-04-23T21:47:45.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:45.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:45.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:47:45.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:47:45.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:47:45.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:47:45.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_34 2024-04-23T21:47:45.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:47:45.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:47:45.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:45.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:45.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:47:45.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:47:45.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:45.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:47:45.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:45.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:47:45.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:45.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_34 2024-04-23T21:47:45.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:47:45.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:45.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:45.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:45.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:45.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:47:45.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:45.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:45.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:45.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:45.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:47:45.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:45.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:47:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:47:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:47:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:47:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:47:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:47:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:47:45.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:47:45.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:47:45.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:47:45.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:45.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:45.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:45.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:45.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_34 2024-04-23T21:47:45.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_34'] 2024-04-23T21:47:45.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:47:45.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:45.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_34 2024-04-23T21:47:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:47:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:47:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:47:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:47:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:45.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:45.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:45.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:47:45.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:47:45.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:47:45.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:45.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:47:45.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:47:45.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:47:45.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:47:45.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:45.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:45.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:47:45.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:47:45.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:47:45.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:47:45.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:45.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:45.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:45.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:45.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_34 2024-04-23T21:47:45.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:47:45.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:47:45.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:47:45.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:47:45.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:47:45.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:47:45.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:47:45.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:45.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_34 2024-04-23T21:47:45.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:47:45.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:47:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:47:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:47:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_34 2024-04-23T21:47:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:47:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:47:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:47:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:47:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:47:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:47:45.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:45.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:47:45.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_34 2024-04-23T21:47:45.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:45.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:45.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:47:45.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:45.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_34 ']' 2024-04-23T21:47:45.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_34' 2024-04-23T21:47:45.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_34 2024-04-23T21:47:45.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:45.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:45.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:45.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:45.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:45.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:45.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:45.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:45.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:45.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:45.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_34 2024-04-23T21:47:45.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:45.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:45.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:45.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:45.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:45.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:45.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:45.871 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:45.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:45.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:45.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:45.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:45.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:45.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:45.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:45.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:45.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:45.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:45.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:45.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:45.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:45.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:45.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:45.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:45.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:45.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:45.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:45.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:45.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:45.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:45.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:45.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:45.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:45.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_34 2024-04-23T21:47:45.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_34'] 2024-04-23T21:47:45.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:45.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_34 2024-04-23T21:47:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:45.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:45.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:45.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:45.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:45.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:45.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:45.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:45.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:45.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:45.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:45.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:45.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:45.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:45.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:45.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:45.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:45.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:45.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:45.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:45.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:45.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:45.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:45.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_34 2024-04-23T21:47:46.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:46.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_34 2024-04-23T21:47:46.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:46.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:46.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:46.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_34 '' 2024-04-23T21:47:46.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:46.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:46.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:46.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:46.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_34.status 2024-04-23T21:47:46.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_34.status 2024-04-23T21:47:46.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:46.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:46.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_34 2024-04-23T21:47:46.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:46.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:46.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:46.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:46.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:46.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=16197], [object_number=12, tag_tid=1, entry_tid=16196], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:46.134 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:46.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:46.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_34 2024-04-23T21:47:46.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:46.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:46.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:46.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_34 '..*' 2024-04-23T21:47:46.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:46.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:46.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:46.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:46.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_34.status 2024-04-23T21:47:46.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_34.status 2024-04-23T21:47:46.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:46.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:46.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_34 2024-04-23T21:47:46.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:46.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:46.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:46.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:46.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:46.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=16197], [object_number=12, tag_tid=1, entry_tid=16196], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:46.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:46.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=13, tag_tid=1, entry_tid=16197' '!=' '' 2024-04-23T21:47:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:47:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=13, tag_tid=1, entry_tid=16197' = '' 2024-04-23T21:47:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:47:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:47:46.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:47:46.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:46.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:47:46.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:47:46.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:47:46.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:46.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:47:46.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:47:46.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:47:46.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:46.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:47:46.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:47:46.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=16197 2024-04-23T21:47:46.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 16197 2024-04-23T21:47:46.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:47:46.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:46.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:47:46.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_34 2024-04-23T21:47:46.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:46.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:46.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:47:46.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:46.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_34 ']' 2024-04-23T21:47:46.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_34' 2024-04-23T21:47:46.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_34 2024-04-23T21:47:46.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:46.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:46.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:46.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:46.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:46.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:46.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:46.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:46.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:46.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:46.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_34 2024-04-23T21:47:46.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:46.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:46.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:46.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:46.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:46.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:46.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:46.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:46.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:46.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:46.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:46.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:46.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:46.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:46.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:46.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:46.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:46.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:46.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:46.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:46.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:46.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:46.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:46.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:46.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:46.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:46.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:46.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:46.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:46.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:46.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:46.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:46.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:46.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_34 2024-04-23T21:47:46.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_34'] 2024-04-23T21:47:46.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:46.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:46.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:46.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_34 2024-04-23T21:47:46.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:46.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:46.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:46.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:46.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:46.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:46.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:46.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:46.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:46.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:46.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:46.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:46.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:46.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:46.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:46.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:46.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:46.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:46.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:46.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:46.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:46.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:46.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:46.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:46.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:46.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:46.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:46.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:46.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:46.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:46.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:46.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:46.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:46.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_34 2024-04-23T21:47:46.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:46.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_34 2024-04-23T21:47:46.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:46.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:46.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:46.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_34 '' 2024-04-23T21:47:46.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:46.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:46.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:46.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:46.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_34.status 2024-04-23T21:47:46.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_34.status 2024-04-23T21:47:46.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:46.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:46.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_34 2024-04-23T21:47:46.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:46.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:46.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:46.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:46.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:46.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=16197], [object_number=12, tag_tid=1, entry_tid=16196], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:46.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:46.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:46.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_34 2024-04-23T21:47:46.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:46.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:46.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:46.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_34 '..*' 2024-04-23T21:47:46.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:46.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:46.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:46.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:46.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_34.status 2024-04-23T21:47:46.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_34.status 2024-04-23T21:47:46.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:46.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:46.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_34 2024-04-23T21:47:46.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:46.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:46.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:46.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:46.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:46.810 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=16197], [object_number=12, tag_tid=1, entry_tid=16196], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:46.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:46.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:46.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:46.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=13, tag_tid=1, entry_tid=16197' '!=' 'object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:46.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:46.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:47:47.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_34 2024-04-23T21:47:47.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:47.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:47.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:47:47.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:47.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_34 ']' 2024-04-23T21:47:47.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_34' 2024-04-23T21:47:47.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_34 2024-04-23T21:47:47.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:47.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:47.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:47.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:47.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:47.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:47.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:47.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:47.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:47.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:47.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_34 2024-04-23T21:47:47.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:47.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:47.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:47.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:47.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:47.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:47.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:47.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:47.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:47.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:47.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:47.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:47.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:47.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:47.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:47.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:47.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:47.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:47.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:47.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:47.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:47.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:47.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:47.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:47.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:47.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:47.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:47.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:47.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:47.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:47.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:47.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:47.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:47.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_34 2024-04-23T21:47:47.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_34'] 2024-04-23T21:47:47.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:47.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:47.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:47.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_34 2024-04-23T21:47:47.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:47.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:47.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:47.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:47.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:47.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:47.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:47.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:47.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:47.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:47.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:47.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:47.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:47.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:47.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:47.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:47.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:47.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:47.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:47.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_34 2024-04-23T21:47:47.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:47.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_34 2024-04-23T21:47:47.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_34 '' 2024-04-23T21:47:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_34.status 2024-04-23T21:47:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_34.status 2024-04-23T21:47:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:47.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:47.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_34 2024-04-23T21:47:47.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:47.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:47.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:47.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:47.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:47.483 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=16197], [object_number=12, tag_tid=1, entry_tid=16196], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:47.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:47.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:47.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_34 2024-04-23T21:47:47.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:47.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:47.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:47.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_34 '..*' 2024-04-23T21:47:47.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:47.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:47.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:47.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:47.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_34.status 2024-04-23T21:47:47.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_34.status 2024-04-23T21:47:47.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:47.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:47.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_34 2024-04-23T21:47:47.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:47.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:47.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:47.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:47.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:47.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=16197], [object_number=12, tag_tid=1, entry_tid=16196], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:47.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:47.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:47.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:47.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=13, tag_tid=1, entry_tid=16197' '!=' 'object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:47.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:47.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:47:48.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_34 2024-04-23T21:47:48.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:48.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:48.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:47:48.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:48.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_34 ']' 2024-04-23T21:47:48.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_34' 2024-04-23T21:47:48.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_34 2024-04-23T21:47:48.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:48.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:48.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:48.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:48.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:48.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:48.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:48.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:48.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:48.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:48.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_34 2024-04-23T21:47:48.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:48.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:48.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:48.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:48.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:48.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:48.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:48.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:48.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:48.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:48.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:48.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:48.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:48.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:48.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:48.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:48.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:48.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:48.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:48.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:48.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:48.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:48.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:48.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:48.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:48.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:48.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:48.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:48.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:48.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:48.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:48.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:48.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:48.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_34 2024-04-23T21:47:48.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_34'] 2024-04-23T21:47:48.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:48.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:48.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:48.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_34 2024-04-23T21:47:48.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:48.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:48.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:48.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:48.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:48.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:48.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:48.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:48.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:48.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:48.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:48.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:48.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:48.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:48.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:48.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:48.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:48.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:48.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:48.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:48.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_34 2024-04-23T21:47:48.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:48.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_34 2024-04-23T21:47:48.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:48.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_34 '' 2024-04-23T21:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_34.status 2024-04-23T21:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_34.status 2024-04-23T21:47:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:48.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:48.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_34 2024-04-23T21:47:48.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:48.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:48.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:48.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:48.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:48.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=16197], [object_number=12, tag_tid=1, entry_tid=16196], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:48.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:48.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:48.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_34 2024-04-23T21:47:48.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_34 '..*' 2024-04-23T21:47:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_34.status 2024-04-23T21:47:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_34.status 2024-04-23T21:47:48.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:48.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:48.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_34 2024-04-23T21:47:48.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:48.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:48.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:48.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:48.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:48.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=16197], [object_number=12, tag_tid=1, entry_tid=16196], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:48.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:48.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:48.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:48.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=13, tag_tid=1, entry_tid=16197' '!=' 'object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:48.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:48.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:47:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_34 2024-04-23T21:47:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:47:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_34 ']' 2024-04-23T21:47:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_34' 2024-04-23T21:47:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_34 2024-04-23T21:47:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:50.249 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:50.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:50.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:50.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:50.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:50.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_34 2024-04-23T21:47:50.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:50.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:50.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:50.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:50.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:50.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:50.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:50.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:50.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:50.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:50.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:50.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:50.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:50.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:50.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:50.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:50.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:50.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:50.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:50.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:50.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:50.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:50.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:50.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:50.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:50.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:50.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:50.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:50.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:50.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:50.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:50.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:50.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:50.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_34 2024-04-23T21:47:50.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_34'] 2024-04-23T21:47:50.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:50.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:50.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:50.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_34 2024-04-23T21:47:50.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:50.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:50.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:50.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:50.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:50.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:50.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:50.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:50.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:50.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:50.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:50.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:50.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:50.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:50.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:50.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:50.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:50.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:50.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:50.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:50.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:50.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:50.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:50.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:50.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:50.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:50.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:50.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:50.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:50.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:50.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:50.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:50.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:50.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_34 2024-04-23T21:47:50.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:50.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_34 2024-04-23T21:47:50.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:50.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:50.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:50.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_34 '' 2024-04-23T21:47:50.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:50.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:50.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:50.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:50.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_34.status 2024-04-23T21:47:50.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_34.status 2024-04-23T21:47:50.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:50.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:50.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_34 2024-04-23T21:47:50.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:50.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:50.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:50.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:50.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:50.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=16197], [object_number=12, tag_tid=1, entry_tid=16196], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:50.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:50.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_34 2024-04-23T21:47:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_34 '..*' 2024-04-23T21:47:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_34.status 2024-04-23T21:47:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_34.status 2024-04-23T21:47:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_34 2024-04-23T21:47:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:50.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:50.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:50.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:50.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:50.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=16197], [object_number=12, tag_tid=1, entry_tid=16196], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:50.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=13, tag_tid=1, entry_tid=16197' '!=' 'object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:47:52.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_34 2024-04-23T21:47:52.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:52.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:52.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:47:52.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:52.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_34 ']' 2024-04-23T21:47:52.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_34' 2024-04-23T21:47:52.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_34 2024-04-23T21:47:52.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:52.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:52.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:52.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:52.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:52.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:52.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:52.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:52.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:52.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:52.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_34 2024-04-23T21:47:52.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:52.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:52.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:52.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:52.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:52.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:52.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:52.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:52.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:52.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:52.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:52.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:52.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:52.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:52.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:52.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:52.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:52.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:52.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_34 2024-04-23T21:47:52.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_34'] 2024-04-23T21:47:52.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:52.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:52.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:52.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_34 2024-04-23T21:47:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:52.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:52.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:52.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:52.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:52.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:52.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_34 2024-04-23T21:47:52.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:52.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_34 2024-04-23T21:47:52.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:52.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:52.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:52.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_34 '' 2024-04-23T21:47:52.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:52.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:52.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:52.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:52.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_34.status 2024-04-23T21:47:52.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_34.status 2024-04-23T21:47:52.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:52.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:52.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_34 2024-04-23T21:47:52.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:52.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:52.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:52.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:52.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:52.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=16197], [object_number=12, tag_tid=1, entry_tid=16196], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:52.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:52.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:52.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_34 2024-04-23T21:47:52.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:52.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:52.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:52.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_34 '..*' 2024-04-23T21:47:52.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:52.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:52.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:52.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:52.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_34.status 2024-04-23T21:47:52.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_34.status 2024-04-23T21:47:52.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:52.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:52.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_34 2024-04-23T21:47:52.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:52.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=1, entry_tid=16197], [object_number=12, tag_tid=1, entry_tid=16196], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:52.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:52.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:52.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:52.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=13, tag_tid=1, entry_tid=16197' '!=' 'object_number=13, tag_tid=1, entry_tid=16197' 2024-04-23T21:47:52.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:47:52.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:47:54.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_34 2024-04-23T21:47:54.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:54.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:54.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:47:54.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:47:54.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_34 ']' 2024-04-23T21:47:54.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_34' 2024-04-23T21:47:54.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_34 2024-04-23T21:47:54.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:47:54.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:54.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:54.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:47:54.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:47:54.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:54.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:47:54.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:47:54.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:47:54.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:54.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_34 2024-04-23T21:47:54.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:47:54.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:54.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:54.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:47:54.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:47:54.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:54.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:54.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:54.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:47:54.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:54.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:54.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:54.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:47:54.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:54.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:47:54.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:47:54.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:54.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:54.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:47:54.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:47:54.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:54.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:54.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:54.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:47:54.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:47:54.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:54.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:54.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:47:54.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:47:54.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:47:54.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:54.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:54.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:47:54.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_34 2024-04-23T21:47:55.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_34'] 2024-04-23T21:47:55.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:47:55.000 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:47:55.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:47:55.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_34 2024-04-23T21:47:55.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:47:55.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:47:55.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:47:55.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:47:55.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:47:55.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:47:55.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:47:55.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:47:55.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:47:55.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:55.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:47:55.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:55.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:47:55.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:55.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:47:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:47:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:47:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:47:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:47:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:47:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:47:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:47:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:47:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:47:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:47:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:47:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:47:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:47:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:47:55.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:47:55.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:55.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:55.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:47:55.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_34 2024-04-23T21:47:55.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:55.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_34 2024-04-23T21:47:55.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:55.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:55.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:55.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_34 '' 2024-04-23T21:47:55.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:55.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:55.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:55.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:47:55.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_34.status 2024-04-23T21:47:55.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_34.status 2024-04-23T21:47:55.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:55.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:55.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_34 2024-04-23T21:47:55.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:55.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:55.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:55.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:55.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:55.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:55.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:55.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_34 2024-04-23T21:47:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_34 '..*' 2024-04-23T21:47:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:47:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:47:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:47:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:47:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_34.status 2024-04-23T21:47:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_34.status 2024-04-23T21:47:55.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:47:55.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:55.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_34 2024-04-23T21:47:55.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:55.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:47:55.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:47:55.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:47:55.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:55.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=16195], [object_number=14, tag_tid=1, entry_tid=16194]]], state=connected] 2024-04-23T21:47:55.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.status 2024-04-23T21:47:55.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:55.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:47:55.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:55.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_34 snap 2024-04-23T21:47:55.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:47:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:47:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:47:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:47:55.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:47:55.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.48438144435582464) > weight (0.25) 2024-04-23T21:47:55.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5502213341097365) > weight (0.25) 2024-04-23T21:47:55.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6574478998254337) > weight (0.25) 2024-04-23T21:47:55.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8871524563556535) > weight (0.25) 2024-04-23T21:47:55.907 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2024-04-23T21:47:56.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_34@snap 2024-04-23T21:47:56.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_34': 2024-04-23T21:47:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:47:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:47:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:47:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 116eb73f4f8e 2024-04-23T21:47:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.116eb73f4f8e 2024-04-23T21:47:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:47:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:47:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:47:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:47:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:37:22 2024 2024-04-23T21:47:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:37:22 2024 2024-04-23T21:47:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:39:24 2024 2024-04-23T21:47:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:47:56.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 116eb73f4f8e 2024-04-23T21:47:56.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:47:56.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:47:56.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: d6b10407-4281-48b3-b11d-c042492f0ea7 2024-04-23T21:47:56.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:47:56.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:56.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_34 snap 2024-04-23T21:47:56.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:56.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:47:56.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:47:56.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:47:56.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.export 2024-04-23T21:47:56.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_34.export 2024-04-23T21:47:56.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_34.export 2024-04-23T21:47:56.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_34@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.export 2024-04-23T21:47:56.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:47:56.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_34@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_34.export 2024-04-23T21:47:56.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:47:56.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_34.export 2024-04-23T21:47:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_34.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_34.export 2024-04-23T21:47:56.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:47:56.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:47:56.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_35 2024-04-23T21:47:56.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_35 snap 2024-04-23T21:47:56.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:47:56.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:56.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:47:56.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:47:56.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_35@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:47:58.197 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:47:58.197+0000 7f83fbfb5640 -1 reset not still connected to 0x562a6c112b60 2024-04-23T21:47:58.197 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:47:58.197+0000 7f83fbfb5640 -1 reset not still connected to 0x562a7184cc30 2024-04-23T21:47:58.197 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:47:58.197+0000 7f83fbfb5640 -1 reset not still connected to 0x562a7a8151e0 2024-04-23T21:47:58.198 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:47:58.198+0000 7f83fbfb5640 -1 reset not still connected to 0x562a7bf94000 2024-04-23T21:47:58.198 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:47:58.198+0000 7f83fbfb5640 -1 reset not still connected to 0x562a7bf951e0 2024-04-23T21:47:58.198 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:47:58.198+0000 7f83fbfb5640 -1 reset not still connected to 0x562a6c112b60 2024-04-23T21:47:58.198 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:47:58.198+0000 7f83fbfb5640 -1 reset not still connected to 0x562a7a8151e0 2024-04-23T21:47:58.198 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:47:58.198+0000 7f83fbfb5640 -1 reset not still connected to 0x562a7bf951e0 2024-04-23T21:47:59.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_35 2024-04-23T21:47:59.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:59.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:59.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:47:59.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_35 started 2024-04-23T21:47:59.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:47:59.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:47:59.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:47:59.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:47:59.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:47:59.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:47:59.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:48:00.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_35 started 2024-04-23T21:48:00.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:00.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:00.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:48:00.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:48:00.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:48:00.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:48:00.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_35 2024-04-23T21:48:00.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:48:00.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:48:00.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:00.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:00.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:48:00.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:48:00.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:00.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:48:00.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:00.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:48:00.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:00.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_35 2024-04-23T21:48:00.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:48:00.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:00.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:00.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:00.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:00.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:48:00.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:00.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:00.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:00.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:00.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:48:00.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:00.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:00.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:00.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:00.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:48:00.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:48:00.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:48:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:48:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:48:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:48:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:48:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:48:00.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:48:00.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:48:00.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:00.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:00.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_35 2024-04-23T21:48:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_35'] 2024-04-23T21:48:00.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:48:00.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:00.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:00.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_35 2024-04-23T21:48:00.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:48:00.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:00.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:00.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:00.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:48:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:48:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:48:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:48:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:48:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:00.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:48:00.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:48:00.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:48:00.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:00.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:00.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:00.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:48:00.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:48:00.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:48:00.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:48:00.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:00.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:00.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:00.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:00.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_35 2024-04-23T21:48:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_35'] 2024-04-23T21:48:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:48:00.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:00.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:00.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_35 2024-04-23T21:48:00.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:48:00.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:00.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:00.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:00.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:00.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:48:00.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:00.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:00.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:00.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:00.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:48:00.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:00.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:00.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:00.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:00.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:48:00.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:48:00.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:48:00.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:00.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:48:00.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:48:00.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:48:00.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:00.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:00.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:00.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:48:00.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:48:00.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:48:00.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:48:00.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:00.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:00.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:00.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:00.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_35 2024-04-23T21:48:00.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:48:00.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:48:00.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:48:00.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:48:00.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:48:00.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:48:00.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:48:00.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:00.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_35 2024-04-23T21:48:00.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:48:00.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:00.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:00.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:48:00.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:48:00.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_35 2024-04-23T21:48:00.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:48:00.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:00.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:00.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:48:00.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:48:00.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:48:00.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:48:00.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:00.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:48:00.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_35 2024-04-23T21:48:00.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:00.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:00.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:48:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_35 ']' 2024-04-23T21:48:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_35' 2024-04-23T21:48:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_35 2024-04-23T21:48:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:00.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:00.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:00.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_35 2024-04-23T21:48:00.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:00.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:00.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:00.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:00.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:00.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:00.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:00.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:00.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:00.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:00.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:00.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:00.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:00.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:00.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:00.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:00.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:00.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:00.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:00.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:01.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:01.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:01.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:01.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:01.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_35'] 2024-04-23T21:48:01.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:01.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:01.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:01.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_35 2024-04-23T21:48:01.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:01.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:01.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:01.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:01.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:01.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:01.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:01.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:01.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:01.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:01.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:01.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:01.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:01.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:01.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:01.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:01.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:01.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:01.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:01.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:01.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_35'] 2024-04-23T21:48:01.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:01.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:01.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:01.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_35 2024-04-23T21:48:01.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:01.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:01.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:01.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:01.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:01.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:01.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:01.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:01.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:01.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_35 2024-04-23T21:48:01.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:01.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:01.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:01.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_35 '' 2024-04-23T21:48:01.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:01.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:01.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:01.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:01.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_35.status 2024-04-23T21:48:01.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_35.status 2024-04-23T21:48:01.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:01.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:01.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_35 2024-04-23T21:48:01.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:01.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:48:01.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:48:01.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:01.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=14522], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:01.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=14522], [object_number=13, tag_tid=1, entry_tid=14521], [object_number=12, tag_tid=1, entry_tid=14520], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:01.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:01.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:48:01.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_35 2024-04-23T21:48:01.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:01.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:01.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:01.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_35 '..*' 2024-04-23T21:48:01.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:01.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:01.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:01.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:01.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_35.status 2024-04-23T21:48:01.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_35.status 2024-04-23T21:48:01.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:01.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:01.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_35 2024-04-23T21:48:01.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:48:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:48:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=14522], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=14522], [object_number=13, tag_tid=1, entry_tid=14521], [object_number=12, tag_tid=1, entry_tid=14520], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=14, tag_tid=1, entry_tid=14522' '!=' '' 2024-04-23T21:48:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:48:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=14, tag_tid=1, entry_tid=14522' = '' 2024-04-23T21:48:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:48:01.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:48:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:48:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:48:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:48:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:48:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:48:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:01.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:48:01.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:01.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:48:01.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:01.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=14522 2024-04-23T21:48:01.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 14522 2024-04-23T21:48:01.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:48:01.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:01.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:48:01.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_35 2024-04-23T21:48:01.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:01.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:01.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:48:01.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:01.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_35 ']' 2024-04-23T21:48:01.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_35' 2024-04-23T21:48:01.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_35 2024-04-23T21:48:01.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:01.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:01.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:01.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:01.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:01.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:01.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:01.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_35 2024-04-23T21:48:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:01.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:01.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:01.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:01.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:01.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:01.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:01.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:01.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:01.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:01.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:01.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:01.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:01.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:01.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_35'] 2024-04-23T21:48:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:01.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_35 2024-04-23T21:48:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:01.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:01.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:01.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:01.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:01.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:01.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:01.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:01.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:01.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:01.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:01.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:01.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:01.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:01.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:01.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:01.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:01.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:01.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:01.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_35'] 2024-04-23T21:48:01.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:01.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:01.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:01.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_35 2024-04-23T21:48:01.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:01.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:01.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:01.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:01.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:01.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:01.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:01.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:01.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:01.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:01.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:01.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:01.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:01.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:01.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:01.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:01.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:01.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:01.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:01.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:01.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:01.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_35 2024-04-23T21:48:01.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:01.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_35 '' 2024-04-23T21:48:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_35.status 2024-04-23T21:48:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_35.status 2024-04-23T21:48:01.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:01.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:01.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_35 2024-04-23T21:48:01.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:01.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:48:01.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:48:01.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:01.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=14522], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:01.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=14522], [object_number=13, tag_tid=1, entry_tid=14521], [object_number=12, tag_tid=1, entry_tid=14520], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:01.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:01.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:48:01.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_35 2024-04-23T21:48:01.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:01.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:01.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:01.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_35 '..*' 2024-04-23T21:48:01.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:01.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:01.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:01.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:01.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_35.status 2024-04-23T21:48:01.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_35.status 2024-04-23T21:48:01.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:01.968 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:01.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_35 2024-04-23T21:48:01.969 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:02.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:48:02.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:48:02.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:02.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=14522], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:02.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=14522], [object_number=13, tag_tid=1, entry_tid=14521], [object_number=12, tag_tid=1, entry_tid=14520], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:02.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:02.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:02.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:02.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=14, tag_tid=1, entry_tid=14522' '!=' 'object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:02.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:02.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:48:02.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_35 2024-04-23T21:48:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:48:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_35 ']' 2024-04-23T21:48:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_35' 2024-04-23T21:48:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_35 2024-04-23T21:48:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_35 2024-04-23T21:48:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:02.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:02.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:02.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:02.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:02.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:02.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:02.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:02.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:02.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:02.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:02.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:02.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_35'] 2024-04-23T21:48:02.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:02.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:02.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_35 2024-04-23T21:48:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:02.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:02.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:02.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:02.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:02.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:02.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:02.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:02.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:02.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:02.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:02.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:02.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_35'] 2024-04-23T21:48:02.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:02.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:02.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:02.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_35 2024-04-23T21:48:02.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:02.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:02.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:02.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:02.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:02.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:02.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:02.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:02.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:02.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:02.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:02.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:02.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:02.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:02.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:02.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:02.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:02.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:02.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:02.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_35 2024-04-23T21:48:02.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:02.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:02.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:02.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_35 '' 2024-04-23T21:48:02.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:02.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:02.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:02.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:02.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_35.status 2024-04-23T21:48:02.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_35.status 2024-04-23T21:48:02.729 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:02.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:02.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_35 2024-04-23T21:48:02.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:02.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:48:02.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:48:02.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:02.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=14522], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:02.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=14522], [object_number=13, tag_tid=1, entry_tid=14521], [object_number=12, tag_tid=1, entry_tid=14520], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:02.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:02.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:48:02.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_35 2024-04-23T21:48:02.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:02.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:02.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:02.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_35 '..*' 2024-04-23T21:48:02.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:02.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:02.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:02.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:02.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_35.status 2024-04-23T21:48:02.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_35.status 2024-04-23T21:48:02.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:02.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:02.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_35 2024-04-23T21:48:02.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:02.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:48:02.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:48:02.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:02.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=14522], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:02.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=14522], [object_number=13, tag_tid=1, entry_tid=14521], [object_number=12, tag_tid=1, entry_tid=14520], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:02.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:02.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:02.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:02.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=14, tag_tid=1, entry_tid=14522' '!=' 'object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:02.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:02.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:48:03.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_35 2024-04-23T21:48:03.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:03.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:03.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:48:03.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:03.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_35 ']' 2024-04-23T21:48:03.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_35' 2024-04-23T21:48:03.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_35 2024-04-23T21:48:03.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:03.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:03.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:03.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:03.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:03.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:03.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:03.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:03.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:03.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:03.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_35 2024-04-23T21:48:03.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:03.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:03.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:03.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:03.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:03.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:03.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:03.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:03.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:03.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:03.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:03.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:03.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:03.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:03.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:03.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:03.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:03.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:03.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_35'] 2024-04-23T21:48:03.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:03.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:03.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:03.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_35 2024-04-23T21:48:03.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:03.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:03.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:03.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:03.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:03.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:03.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:03.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:03.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:03.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:03.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:03.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:03.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:03.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:03.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:03.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:03.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:03.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:03.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:03.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:03.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:03.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:03.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_35'] 2024-04-23T21:48:03.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:03.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:03.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:03.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_35 2024-04-23T21:48:03.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:03.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:03.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:03.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:03.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:03.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:03.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:03.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:03.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:03.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:03.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:03.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:03.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:03.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:03.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:03.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:03.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:03.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:03.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:03.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:03.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:03.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:03.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:03.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:03.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:03.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:03.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:03.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:03.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:03.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:03.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:03.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:03.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:03.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:03.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:03.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_35 2024-04-23T21:48:03.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:03.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:03.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_35 '' 2024-04-23T21:48:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_35.status 2024-04-23T21:48:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_35.status 2024-04-23T21:48:03.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_35 2024-04-23T21:48:03.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:03.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:48:03.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:48:03.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:03.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=14522], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:03.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=14522], [object_number=13, tag_tid=1, entry_tid=14521], [object_number=12, tag_tid=1, entry_tid=14520], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:03.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:03.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:48:03.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_35 2024-04-23T21:48:03.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:03.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:03.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:03.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_35 '..*' 2024-04-23T21:48:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_35.status 2024-04-23T21:48:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_35.status 2024-04-23T21:48:03.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:03.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:03.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_35 2024-04-23T21:48:03.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:04.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:48:04.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:48:04.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:04.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=14522], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:04.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=14522], [object_number=13, tag_tid=1, entry_tid=14521], [object_number=12, tag_tid=1, entry_tid=14520], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:04.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=14, tag_tid=1, entry_tid=14522' '!=' 'object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:04.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:48:05.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_35 2024-04-23T21:48:05.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:05.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:05.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:48:05.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:05.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_35 ']' 2024-04-23T21:48:05.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_35' 2024-04-23T21:48:05.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_35 2024-04-23T21:48:05.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:05.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:05.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:05.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:05.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:05.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:05.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:05.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:05.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:05.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:05.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_35 2024-04-23T21:48:05.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:05.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:05.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:05.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:05.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:05.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:05.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:05.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:05.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:05.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:05.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:05.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:05.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:05.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:05.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:05.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:05.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:05.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:05.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:05.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:05.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:05.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:05.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:05.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:05.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:05.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:05.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:05.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:05.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:05.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:05.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_35'] 2024-04-23T21:48:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:05.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:05.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:05.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_35 2024-04-23T21:48:05.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:05.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:05.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:05.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:05.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:05.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:05.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:05.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:05.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:05.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:05.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:05.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:05.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:05.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:05.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:05.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:05.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:05.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:05.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:05.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:05.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:05.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:05.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:05.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:05.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:05.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:05.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:05.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:05.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:05.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:05.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:05.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:05.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:05.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:06.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_35'] 2024-04-23T21:48:06.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:06.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:06.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:06.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_35 2024-04-23T21:48:06.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:06.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:06.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:06.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:06.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:06.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:06.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:06.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:06.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:06.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:06.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:06.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:06.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:06.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:06.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:06.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:06.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:06.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:06.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:06.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:06.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:06.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:06.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:06.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:06.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:06.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:06.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:06.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:06.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:06.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:06.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:06.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:06.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:06.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_35 2024-04-23T21:48:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_35 '' 2024-04-23T21:48:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_35.status 2024-04-23T21:48:06.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_35.status 2024-04-23T21:48:06.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:06.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:06.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_35 2024-04-23T21:48:06.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:06.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:48:06.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:48:06.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:06.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=14522], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:06.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=14522], [object_number=13, tag_tid=1, entry_tid=14521], [object_number=12, tag_tid=1, entry_tid=14520], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:06.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:06.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:48:06.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_35 2024-04-23T21:48:06.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:06.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_35 '..*' 2024-04-23T21:48:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_35.status 2024-04-23T21:48:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_35.status 2024-04-23T21:48:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_35 2024-04-23T21:48:06.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:06.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:48:06.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:48:06.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:06.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=14522], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:06.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=14522], [object_number=13, tag_tid=1, entry_tid=14521], [object_number=12, tag_tid=1, entry_tid=14520], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:06.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:06.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:06.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:06.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=14, tag_tid=1, entry_tid=14522' '!=' 'object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:06.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:06.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:48:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_35 2024-04-23T21:48:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:48:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:08.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_35 ']' 2024-04-23T21:48:08.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_35' 2024-04-23T21:48:08.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_35 2024-04-23T21:48:08.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:08.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:08.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:08.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:08.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:08.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:08.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:08.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_35 2024-04-23T21:48:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:08.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:08.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:08.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:08.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:08.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:08.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:08.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:08.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:08.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:08.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_35'] 2024-04-23T21:48:08.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:08.379 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_35 2024-04-23T21:48:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:08.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:08.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:08.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:08.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:08.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:08.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:08.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:08.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:08.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:08.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:08.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_35'] 2024-04-23T21:48:08.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:08.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_35 2024-04-23T21:48:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:08.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:08.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:08.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:08.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:08.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:08.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:08.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:08.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:08.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:08.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:08.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:08.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:08.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:08.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:08.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:08.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_35 2024-04-23T21:48:08.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:08.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:08.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:08.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_35 '' 2024-04-23T21:48:08.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:08.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:08.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:08.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:08.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_35.status 2024-04-23T21:48:08.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_35.status 2024-04-23T21:48:08.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:08.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:08.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_35 2024-04-23T21:48:08.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:48:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:48:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=14522], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:08.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=14522], [object_number=13, tag_tid=1, entry_tid=14521], [object_number=12, tag_tid=1, entry_tid=14520], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:08.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:08.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:48:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_35 2024-04-23T21:48:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_35 '..*' 2024-04-23T21:48:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:08.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:08.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:08.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_35.status 2024-04-23T21:48:08.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_35.status 2024-04-23T21:48:08.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:08.640 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:08.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_35 2024-04-23T21:48:08.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:08.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:48:08.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:48:08.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:08.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=14522], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:08.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=14, tag_tid=1, entry_tid=14522], [object_number=13, tag_tid=1, entry_tid=14521], [object_number=12, tag_tid=1, entry_tid=14520], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:08.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:08.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:08.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:08.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=14, tag_tid=1, entry_tid=14522' '!=' 'object_number=14, tag_tid=1, entry_tid=14522' 2024-04-23T21:48:08.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:08.688 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:48:10.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_35 2024-04-23T21:48:10.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:10.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:10.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:48:10.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:10.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_35 ']' 2024-04-23T21:48:10.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_35' 2024-04-23T21:48:10.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_35 2024-04-23T21:48:10.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:10.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:10.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:10.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:10.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:10.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:10.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:10.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:10.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:10.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:10.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_35 2024-04-23T21:48:10.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:10.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:10.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:10.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:10.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:10.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:10.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:10.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:10.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:10.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:10.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:10.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:10.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:10.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:10.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:10.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:10.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:10.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:10.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:10.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_35'] 2024-04-23T21:48:10.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:10.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:10.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:10.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_35 2024-04-23T21:48:10.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:10.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:10.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:10.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:10.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:10.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:10.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:10.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:10.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:10.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:10.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:10.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:10.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:10.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:10.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:10.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:10.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:10.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:10.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:10.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:10.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:10.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:10.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:10.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:10.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:10.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:10.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:10.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:10.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:10.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:10.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:10.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:10.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:10.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:10.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_35'] 2024-04-23T21:48:10.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:10.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:10.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:10.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_35 2024-04-23T21:48:10.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:10.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:10.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:10.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:10.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:10.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:10.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:10.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:10.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:10.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_35 2024-04-23T21:48:11.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:11.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_35 2024-04-23T21:48:11.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:11.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:11.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:11.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_35 '' 2024-04-23T21:48:11.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:11.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:11.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:11.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:11.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_35.status 2024-04-23T21:48:11.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_35.status 2024-04-23T21:48:11.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:11.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:11.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_35 2024-04-23T21:48:11.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:11.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:48:11.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:48:11.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:11.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=14522], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:11.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=14522], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:11.071 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:11.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:48:11.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_35 2024-04-23T21:48:11.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:11.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:11.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:11.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_35 '..*' 2024-04-23T21:48:11.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:11.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:11.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:48:11.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:11.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_35.status 2024-04-23T21:48:11.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_35.status 2024-04-23T21:48:11.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:11.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:11.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_35 2024-04-23T21:48:11.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:11.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:48:11.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:48:11.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:11.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=14522], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:11.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=14, tag_tid=1, entry_tid=14522], [object_number=15, tag_tid=1, entry_tid=14519]]], state=connected] 2024-04-23T21:48:11.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.status 2024-04-23T21:48:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:48:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:48:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_35 snap 2024-04-23T21:48:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:48:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:48:11.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:48:11.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:48:11.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:48:12.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_35@snap 2024-04-23T21:48:12.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_35': 2024-04-23T21:48:12.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:48:12.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:48:12.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:48:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 11b813e443e4 2024-04-23T21:48:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.11b813e443e4 2024-04-23T21:48:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:48:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:48:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:48:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:48:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:37:36 2024 2024-04-23T21:48:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:37:36 2024 2024-04-23T21:48:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:39:38 2024 2024-04-23T21:48:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:48:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 11b813e443e4 2024-04-23T21:48:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:48:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:48:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: a7646a0d-f281-4b74-9423-a5cf2b28fa39 2024-04-23T21:48:12.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:48:12.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:12.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_35 snap 2024-04-23T21:48:12.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:12.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:48:12.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:48:12.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:48:12.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.export 2024-04-23T21:48:12.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_35.export 2024-04-23T21:48:12.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_35.export 2024-04-23T21:48:12.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_35@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.export 2024-04-23T21:48:12.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:48:12.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_35@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_35.export 2024-04-23T21:48:12.491 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:48:12.491+0000 7fc3fc09a640 -1 bdev(0x56408206ce00 /var/lib/ceph/osd/cluster1-0/block) aio_submit retries 1 2024-04-23T21:48:12.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:48:12.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_35.export 2024-04-23T21:48:12.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_35.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_35.export 2024-04-23T21:48:12.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:12.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:48:12.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_36 2024-04-23T21:48:12.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_36 snap 2024-04-23T21:48:12.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:12.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:12.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:48:12.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:48:12.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_36@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:48:15.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_36 2024-04-23T21:48:15.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:15.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:15.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:48:15.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_36 started 2024-04-23T21:48:15.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:15.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:15.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:48:15.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:48:15.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:48:15.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:15.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:48:16.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_36 started 2024-04-23T21:48:16.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:16.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:16.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:48:16.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:48:16.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:48:16.019 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:48:16.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_36 2024-04-23T21:48:16.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:48:16.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:48:16.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:16.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:16.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:48:16.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:48:16.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:16.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:48:16.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:16.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:48:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_36 2024-04-23T21:48:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:48:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:48:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:16.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:48:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:48:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:48:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:48:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:48:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:48:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:48:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:16.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:16.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:48:16.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:48:16.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:48:16.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:48:16.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:16.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:16.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:16.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:16.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_36 2024-04-23T21:48:16.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_36'] 2024-04-23T21:48:16.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:48:16.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:16.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_36 2024-04-23T21:48:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:48:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:48:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:48:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:16.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:48:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:48:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:48:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:48:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:48:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:48:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:48:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:48:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:48:16.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:48:16.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:16.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:16.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:16.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:16.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_36 2024-04-23T21:48:16.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:48:16.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:48:16.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:48:16.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:48:16.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:48:16.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:48:16.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:48:16.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:16.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_36 2024-04-23T21:48:16.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:48:16.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:16.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:16.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:48:16.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:48:16.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_36 2024-04-23T21:48:16.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:48:16.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:16.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:16.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:48:16.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:48:16.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:48:16.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:48:16.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:16.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:48:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_36 2024-04-23T21:48:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:48:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_36 ']' 2024-04-23T21:48:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_36' 2024-04-23T21:48:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_36 2024-04-23T21:48:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:16.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:16.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:16.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:16.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:16.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:16.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:16.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_36 2024-04-23T21:48:16.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:16.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:16.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:16.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:16.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:16.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:16.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:16.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:16.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:16.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:16.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:16.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:16.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:16.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:16.460 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:16.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:16.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:16.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:16.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_36 2024-04-23T21:48:16.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_36'] 2024-04-23T21:48:16.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:16.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:16.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:16.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_36 2024-04-23T21:48:16.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:16.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:16.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:16.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:16.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:16.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:16.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:16.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:16.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:16.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:16.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:16.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:16.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:16.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:16.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:16.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:16.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:16.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:16.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:16.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:16.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:16.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:16.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:16.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:16.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:16.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:16.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:16.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:16.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:16.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:16.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:16.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:16.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:16.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_36 2024-04-23T21:48:16.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:16.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_36 2024-04-23T21:48:16.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:16.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:16.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_36 2024-04-23T21:48:16.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_36 '' 2024-04-23T21:48:16.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:16.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:16.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_36 2024-04-23T21:48:16.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:16.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_36.status 2024-04-23T21:48:16.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_36.status 2024-04-23T21:48:16.670 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:16.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:16.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_36 2024-04-23T21:48:16.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:16.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:16.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:16.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:16.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10395], [object_number=10, tag_tid=1, entry_tid=10394]]], state=connected] 2024-04-23T21:48:16.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=8, tag_tid=1, entry_tid=10396], [object_number=11, tag_tid=1, entry_tid=10395], [object_number=10, tag_tid=1, entry_tid=10394], [object_number=9, tag_tid=1, entry_tid=10393]]], state=connected] 2024-04-23T21:48:16.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:16.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:48:16.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_36 2024-04-23T21:48:16.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:16.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:16.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_36 2024-04-23T21:48:16.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_36 '..*' 2024-04-23T21:48:16.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:16.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:16.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_36 2024-04-23T21:48:16.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:16.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_36.status 2024-04-23T21:48:16.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_36.status 2024-04-23T21:48:16.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:16.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:16.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_36 2024-04-23T21:48:16.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:16.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:16.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:16.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:16.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10395], [object_number=10, tag_tid=1, entry_tid=10394]]], state=connected] 2024-04-23T21:48:16.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=8, tag_tid=1, entry_tid=10396], [object_number=11, tag_tid=1, entry_tid=10395], [object_number=10, tag_tid=1, entry_tid=10394], [object_number=9, tag_tid=1, entry_tid=10393]]], state=connected] 2024-04-23T21:48:16.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:16.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=8, tag_tid=1, entry_tid=10396' 2024-04-23T21:48:16.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=8, tag_tid=1, entry_tid=10396' 2024-04-23T21:48:16.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=8, tag_tid=1, entry_tid=10396' '!=' '' 2024-04-23T21:48:16.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:48:16.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=8, tag_tid=1, entry_tid=10396' = '' 2024-04-23T21:48:16.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=8, tag_tid=1, entry_tid=10396' 2024-04-23T21:48:16.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:48:16.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:48:16.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:16.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:48:16.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=8, tag_tid=1, entry_tid=10396' 2024-04-23T21:48:16.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:48:16.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:16.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:48:16.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:48:16.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:48:16.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:16.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:48:16.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=8, tag_tid=1, entry_tid=10396' 2024-04-23T21:48:16.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:48:16.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:16.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=10396 2024-04-23T21:48:16.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 10396 2024-04-23T21:48:16.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:48:16.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:16.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:48:16.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_36 2024-04-23T21:48:16.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:16.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:16.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:48:16.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:16.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_36 ']' 2024-04-23T21:48:16.971 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_36' 2024-04-23T21:48:16.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_36 2024-04-23T21:48:16.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:16.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:16.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:16.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:16.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:16.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:16.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:16.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:16.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:16.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:16.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_36 2024-04-23T21:48:16.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:16.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:16.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:16.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:16.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:16.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:16.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:16.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:16.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:16.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:16.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_36 2024-04-23T21:48:17.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_36'] 2024-04-23T21:48:17.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:17.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:17.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:17.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_36 2024-04-23T21:48:17.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:17.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:17.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:17.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:17.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:17.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:17.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:17.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:17.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:17.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:17.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:17.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:17.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:17.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:17.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:17.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:17.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:17.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:17.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:17.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_36 2024-04-23T21:48:17.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:17.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_36 2024-04-23T21:48:17.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:17.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:17.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_36 2024-04-23T21:48:17.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_36 '' 2024-04-23T21:48:17.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:17.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:17.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_36 2024-04-23T21:48:17.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:17.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_36.status 2024-04-23T21:48:17.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_36.status 2024-04-23T21:48:17.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:17.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:17.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_36 2024-04-23T21:48:17.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:17.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:17.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:17.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:17.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10395], [object_number=10, tag_tid=1, entry_tid=10394]]], state=connected] 2024-04-23T21:48:17.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=8, tag_tid=1, entry_tid=10396], [object_number=11, tag_tid=1, entry_tid=10395], [object_number=10, tag_tid=1, entry_tid=10394], [object_number=9, tag_tid=1, entry_tid=10393]]], state=connected] 2024-04-23T21:48:17.227 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:17.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:48:17.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_36 2024-04-23T21:48:17.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:17.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:17.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_36 2024-04-23T21:48:17.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_36 '..*' 2024-04-23T21:48:17.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:17.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:17.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_36 2024-04-23T21:48:17.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:17.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_36.status 2024-04-23T21:48:17.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_36.status 2024-04-23T21:48:17.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:17.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:17.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_36 2024-04-23T21:48:17.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:17.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:17.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:17.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:17.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10395], [object_number=10, tag_tid=1, entry_tid=10394]]], state=connected] 2024-04-23T21:48:17.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=8, tag_tid=1, entry_tid=10396], [object_number=11, tag_tid=1, entry_tid=10395], [object_number=10, tag_tid=1, entry_tid=10394], [object_number=9, tag_tid=1, entry_tid=10393]]], state=connected] 2024-04-23T21:48:17.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:17.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=8, tag_tid=1, entry_tid=10396' 2024-04-23T21:48:17.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=8, tag_tid=1, entry_tid=10396' 2024-04-23T21:48:17.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=8, tag_tid=1, entry_tid=10396' '!=' 'object_number=8, tag_tid=1, entry_tid=10396' 2024-04-23T21:48:17.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:17.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:48:17.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_36 2024-04-23T21:48:17.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:17.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:17.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:48:17.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:17.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_36 ']' 2024-04-23T21:48:17.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_36' 2024-04-23T21:48:17.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_36 2024-04-23T21:48:17.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:17.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:17.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:17.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:17.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:17.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:17.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:17.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:17.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:17.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:17.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_36 2024-04-23T21:48:17.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:17.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:17.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:17.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:17.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:17.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:17.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:17.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:17.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:17.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:17.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:17.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:17.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:17.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:17.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:17.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:17.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:17.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:17.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:17.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_36 2024-04-23T21:48:17.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_36'] 2024-04-23T21:48:17.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:17.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:17.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:17.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_36 2024-04-23T21:48:17.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:17.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:17.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:17.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:17.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:17.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:17.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:17.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:17.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:17.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:17.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:17.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:17.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:17.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:17.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:17.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:17.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:17.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_36 2024-04-23T21:48:17.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:17.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_36 2024-04-23T21:48:17.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:17.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:17.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_36 2024-04-23T21:48:17.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_36 '' 2024-04-23T21:48:17.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:17.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:17.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_36 2024-04-23T21:48:17.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:17.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_36.status 2024-04-23T21:48:17.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_36.status 2024-04-23T21:48:17.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:17.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:17.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_36 2024-04-23T21:48:17.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:17.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:17.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:17.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:17.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10395], [object_number=10, tag_tid=1, entry_tid=10394]]], state=connected] 2024-04-23T21:48:17.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10395], [object_number=10, tag_tid=1, entry_tid=10394]]], state=connected] 2024-04-23T21:48:17.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:48:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_36 2024-04-23T21:48:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_36 2024-04-23T21:48:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_36 '..*' 2024-04-23T21:48:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_36 2024-04-23T21:48:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_36.status 2024-04-23T21:48:17.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_36.status 2024-04-23T21:48:17.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:17.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:17.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_36 2024-04-23T21:48:17.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:17.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:17.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:17.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:17.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10395], [object_number=10, tag_tid=1, entry_tid=10394]]], state=connected] 2024-04-23T21:48:17.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10395], [object_number=10, tag_tid=1, entry_tid=10394]]], state=connected] 2024-04-23T21:48:17.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.status 2024-04-23T21:48:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:48:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:48:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_36 snap 2024-04-23T21:48:17.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:17.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:17.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:48:17.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:48:17.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:48:17.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:48:17.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:48:18.987 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_36@snap 2024-04-23T21:48:19.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_36': 2024-04-23T21:48:19.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:48:19.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:48:19.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:48:19.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 121de1afb95a 2024-04-23T21:48:19.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.121de1afb95a 2024-04-23T21:48:19.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:48:19.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:48:19.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:48:19.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:48:19.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:37:49 2024 2024-04-23T21:48:19.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:37:49 2024 2024-04-23T21:48:19.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:38:50 2024 2024-04-23T21:48:19.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:48:19.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 121de1afb95a 2024-04-23T21:48:19.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:48:19.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:48:19.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: d8666120-1e98-413f-97fc-065941020804 2024-04-23T21:48:19.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:48:19.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:19.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_36 snap 2024-04-23T21:48:19.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:19.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:48:19.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:48:19.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:48:19.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.export 2024-04-23T21:48:19.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_36.export 2024-04-23T21:48:19.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_36.export 2024-04-23T21:48:19.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_36@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.export 2024-04-23T21:48:19.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:48:19.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_36@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_36.export 2024-04-23T21:48:19.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:48:19.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_36.export 2024-04-23T21:48:19.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_36.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_36.export 2024-04-23T21:48:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:48:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_37 2024-04-23T21:48:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_37 snap 2024-04-23T21:48:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:48:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:48:19.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_37@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:48:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_37 2024-04-23T21:48:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:48:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_37 started 2024-04-23T21:48:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:48:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:48:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:48:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:21.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:48:22.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_37 started 2024-04-23T21:48:22.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:22.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:22.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:48:22.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:48:22.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:48:22.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:48:22.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_37 2024-04-23T21:48:22.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:48:22.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:48:22.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:22.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:22.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:48:22.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:48:22.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:22.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:48:22.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:22.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:48:22.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:22.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_37 2024-04-23T21:48:22.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:48:22.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:22.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:22.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:22.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:22.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:48:22.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:22.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:22.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:22.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:22.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:48:22.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:22.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:22.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:22.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:22.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:48:22.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:48:22.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:48:22.036 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:22.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:48:22.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:48:22.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:48:22.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:22.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:22.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:22.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:48:22.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:48:22.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:48:22.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:48:22.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:22.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:22.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:22.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:22.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_37 2024-04-23T21:48:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2024-04-23T21:48:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:48:22.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:22.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:22.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_37 2024-04-23T21:48:22.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:48:22.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:22.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:22.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:22.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:22.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:48:22.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:22.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:48:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:48:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:48:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:48:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:48:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:48:22.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:48:22.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:22.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:22.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:22.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:48:22.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:48:22.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:48:22.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:48:22.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:22.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:22.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:22.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:22.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_37 2024-04-23T21:48:22.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2024-04-23T21:48:22.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:48:22.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:22.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:22.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_37 2024-04-23T21:48:22.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:48:22.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:22.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:22.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:22.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:22.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:48:22.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:22.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:22.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:22.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:22.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:48:22.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:22.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:48:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:48:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:48:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:48:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:48:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:48:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:48:22.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:48:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:48:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:48:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:22.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:22.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:22.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:22.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_37 2024-04-23T21:48:22.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:48:22.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:48:22.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:48:22.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:48:22.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:48:22.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:48:22.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:48:22.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:22.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_37 2024-04-23T21:48:22.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:48:22.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:22.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:22.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:48:22.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:48:22.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_37 2024-04-23T21:48:22.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:48:22.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:22.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:22.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:48:22.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:48:22.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:48:22.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:48:22.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:22.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:48:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_37 2024-04-23T21:48:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:48:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_37 ']' 2024-04-23T21:48:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_37' 2024-04-23T21:48:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_37 2024-04-23T21:48:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:22.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:22.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:22.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:22.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_37 2024-04-23T21:48:22.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:22.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:22.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:22.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:22.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:22.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:22.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:22.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:22.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:22.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:22.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:22.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:22.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:22.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:22.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:22.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:22.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:22.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:22.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:22.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:22.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:22.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:22.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:22.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:22.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:22.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:22.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:22.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:22.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:22.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:22.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:22.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:22.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2024-04-23T21:48:22.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:22.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:22.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:22.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_37 2024-04-23T21:48:22.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:22.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:22.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:22.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:22.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:22.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:22.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:22.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:22.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:22.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:22.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:22.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:22.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:22.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:22.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:22.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:22.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:22.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:22.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:22.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:22.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:22.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:22.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:22.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:22.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:22.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:22.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:22.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:22.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:22.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:22.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:22.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:22.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:22.689 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:22.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2024-04-23T21:48:22.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:22.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_37 2024-04-23T21:48:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:22.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:22.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:22.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:22.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:22.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:22.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:22.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:22.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:22.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:22.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:22.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:22.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:22.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:22.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_37 2024-04-23T21:48:22.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:22.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:22.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:22.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_37 '' 2024-04-23T21:48:22.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:22.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:22.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:22.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:22.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_37.status 2024-04-23T21:48:22.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_37.status 2024-04-23T21:48:22.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:22.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:22.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_37 2024-04-23T21:48:22.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:22.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:22.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:22.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:22.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:22.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6089], [object_number=4, tag_tid=1, entry_tid=6088], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:22.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:22.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:22.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_37 2024-04-23T21:48:22.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:22.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:22.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:22.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_37 '..*' 2024-04-23T21:48:22.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:22.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:22.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:22.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:22.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_37.status 2024-04-23T21:48:22.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_37.status 2024-04-23T21:48:22.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:22.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:22.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_37 2024-04-23T21:48:22.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:22.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:22.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:22.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:22.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:22.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6089], [object_number=4, tag_tid=1, entry_tid=6088], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:22.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:22.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:22.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:22.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=6089' '!=' '' 2024-04-23T21:48:22.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:48:22.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=6089' = '' 2024-04-23T21:48:22.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:22.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:22.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:48:22.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:22.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:48:22.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:22.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:48:22.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:22.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:48:22.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:22.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:48:22.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:22.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:48:22.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:22.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:48:22.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:22.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=6089 2024-04-23T21:48:22.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 6089 2024-04-23T21:48:22.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:48:22.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:22.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:48:23.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_37 2024-04-23T21:48:23.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:23.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:23.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:48:23.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:23.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_37 ']' 2024-04-23T21:48:23.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_37' 2024-04-23T21:48:23.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_37 2024-04-23T21:48:23.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:23.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:23.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:23.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:23.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:23.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:23.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:23.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:23.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:23.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:23.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_37 2024-04-23T21:48:23.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:23.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:23.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:23.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:23.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:23.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:23.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:23.184 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:23.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:23.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:23.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:23.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:23.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2024-04-23T21:48:23.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:23.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_37 2024-04-23T21:48:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:23.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:23.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:23.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:23.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:23.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:23.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:23.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:23.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2024-04-23T21:48:23.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:23.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:23.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:23.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_37 2024-04-23T21:48:23.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:23.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:23.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:23.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:23.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:23.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:23.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:23.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:23.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:23.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:23.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:23.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:23.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:23.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:23.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:23.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:23.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:23.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:23.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:23.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:23.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:23.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:23.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:23.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:23.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:23.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:23.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:23.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:23.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:23.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:23.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:23.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:23.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:23.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:23.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:23.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_37 2024-04-23T21:48:23.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:23.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:23.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:23.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_37 '' 2024-04-23T21:48:23.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:23.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:23.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:23.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:23.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_37.status 2024-04-23T21:48:23.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_37.status 2024-04-23T21:48:23.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:23.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:23.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_37 2024-04-23T21:48:23.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:23.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:23.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:23.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:23.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:23.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6089], [object_number=4, tag_tid=1, entry_tid=6088], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:23.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:23.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:23.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_37 2024-04-23T21:48:23.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:23.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:23.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:23.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_37 '..*' 2024-04-23T21:48:23.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:23.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:23.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:23.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:23.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_37.status 2024-04-23T21:48:23.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_37.status 2024-04-23T21:48:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_37 2024-04-23T21:48:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:23.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:23.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:23.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:23.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:23.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6089], [object_number=4, tag_tid=1, entry_tid=6088], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:23.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:23.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:23.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:23.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=6089' '!=' 'object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:23.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:23.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:48:23.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_37 2024-04-23T21:48:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:48:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_37 ']' 2024-04-23T21:48:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_37' 2024-04-23T21:48:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_37 2024-04-23T21:48:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:23.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:23.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:23.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:23.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:23.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_37 2024-04-23T21:48:23.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:23.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:23.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:23.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:23.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:23.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:23.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:23.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:23.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:23.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:23.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:23.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:23.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:23.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:23.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:23.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:23.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:24.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:24.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:24.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:24.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2024-04-23T21:48:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:24.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:24.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:24.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_37 2024-04-23T21:48:24.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:24.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:24.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:24.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:24.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:24.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:24.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:24.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:24.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:24.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:24.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:24.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:24.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:24.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:24.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:24.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:24.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:24.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:24.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:24.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:24.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2024-04-23T21:48:24.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:24.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:24.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_37 2024-04-23T21:48:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:24.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:24.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:24.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:24.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:24.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:24.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:24.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:24.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_37 2024-04-23T21:48:24.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:24.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:24.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:24.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_37 '' 2024-04-23T21:48:24.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:24.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:24.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:24.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:24.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_37.status 2024-04-23T21:48:24.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_37.status 2024-04-23T21:48:24.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:24.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:24.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_37 2024-04-23T21:48:24.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:24.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:24.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:24.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:24.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:24.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6089], [object_number=4, tag_tid=1, entry_tid=6088], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:24.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:24.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:24.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_37 2024-04-23T21:48:24.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:24.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:24.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:24.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_37 '..*' 2024-04-23T21:48:24.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:24.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:24.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:24.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:24.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_37.status 2024-04-23T21:48:24.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_37.status 2024-04-23T21:48:24.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:24.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:24.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_37 2024-04-23T21:48:24.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:24.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:24.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:24.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:24.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:24.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6089], [object_number=4, tag_tid=1, entry_tid=6088], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:24.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:24.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:24.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:24.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=6089' '!=' 'object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:24.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:24.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:48:25.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_37 2024-04-23T21:48:25.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:25.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:25.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:48:25.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:25.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_37 ']' 2024-04-23T21:48:25.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_37' 2024-04-23T21:48:25.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_37 2024-04-23T21:48:25.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:25.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:25.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:25.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:25.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:25.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:25.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:25.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:25.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:25.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:25.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_37 2024-04-23T21:48:25.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:25.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:25.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:25.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:25.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:25.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:25.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:25.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:25.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:25.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:25.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:25.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:25.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:25.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:25.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:25.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:25.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:25.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:25.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:25.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:25.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:25.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:25.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:25.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2024-04-23T21:48:25.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:25.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:25.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:25.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_37 2024-04-23T21:48:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:25.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:25.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:25.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:25.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:25.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:25.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:25.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:25.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:25.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:25.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:25.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:25.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:25.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:25.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:25.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:25.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:25.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:25.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:25.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:25.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:25.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2024-04-23T21:48:25.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:25.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:25.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:25.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_37 2024-04-23T21:48:25.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:25.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:25.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:25.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:25.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:25.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:25.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:25.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:25.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:25.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:25.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:25.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:25.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:25.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:25.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:25.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:25.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:25.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:25.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:25.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:25.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:25.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:25.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:25.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:25.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:25.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:25.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:25.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_37 2024-04-23T21:48:25.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:25.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:25.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:25.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_37 '' 2024-04-23T21:48:25.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:25.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:25.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:25.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:25.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_37.status 2024-04-23T21:48:25.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_37.status 2024-04-23T21:48:25.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:25.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:25.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_37 2024-04-23T21:48:25.522 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:25.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:25.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:25.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:25.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:25.563 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6089], [object_number=4, tag_tid=1, entry_tid=6088], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:25.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:25.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_37 2024-04-23T21:48:25.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:25.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:25.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:25.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_37 '..*' 2024-04-23T21:48:25.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:25.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:25.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:25.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:25.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_37.status 2024-04-23T21:48:25.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_37.status 2024-04-23T21:48:25.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:25.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:25.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_37 2024-04-23T21:48:25.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:25.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:25.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:25.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:25.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:25.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6089], [object_number=4, tag_tid=1, entry_tid=6088], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:25.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:25.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:25.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:25.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=6089' '!=' 'object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:25.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:25.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:48:26.627 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:48:26.621+0000 7fc40b4f4640 -1 reset not still connected to 0x5640a0324750 2024-04-23T21:48:26.627 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:48:26.623+0000 7fc40b4f4640 -1 reset not still connected to 0x5640acc964e0 2024-04-23T21:48:26.627 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:48:26.623+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b8b276c0 2024-04-23T21:48:26.627 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:48:26.623+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bc35ef70 2024-04-23T21:48:26.627 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:48:26.623+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bd19a750 2024-04-23T21:48:26.627 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:48:26.623+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bed57d40 2024-04-23T21:48:26.627 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:48:26.623+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c17a48f0 2024-04-23T21:48:26.629 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:48:26.627+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c181fad0 2024-04-23T21:48:26.629 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:48:26.627+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c18352b0 2024-04-23T21:48:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_37 2024-04-23T21:48:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:48:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_37 ']' 2024-04-23T21:48:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_37' 2024-04-23T21:48:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_37 2024-04-23T21:48:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:27.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:27.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:27.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:27.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_37 2024-04-23T21:48:27.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:27.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:27.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:27.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:27.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:27.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:27.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:27.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:27.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:27.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:27.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:27.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:27.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2024-04-23T21:48:27.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:27.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:27.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:27.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_37 2024-04-23T21:48:27.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:27.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:27.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:27.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:27.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:27.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:27.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:27.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:27.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:27.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:27.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:27.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:27.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:27.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:27.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:27.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:27.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:27.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:27.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:27.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:27.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:27.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:27.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:27.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:27.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:27.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:27.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:27.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:27.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:27.336 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:27.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:27.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:27.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:27.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:27.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2024-04-23T21:48:27.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:27.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:27.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:27.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_37 2024-04-23T21:48:27.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:27.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:27.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:27.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:27.435 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:27.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:27.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:27.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:27.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:27.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:27.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:27.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:27.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:27.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:27.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:27.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:27.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:27.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:27.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:27.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:27.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:27.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:27.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_37 2024-04-23T21:48:27.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:27.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:27.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:27.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_37 '' 2024-04-23T21:48:27.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:27.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:27.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:27.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:27.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_37.status 2024-04-23T21:48:27.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_37.status 2024-04-23T21:48:27.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:27.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:27.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_37 2024-04-23T21:48:27.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:27.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:27.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:27.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:27.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:27.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6089], [object_number=4, tag_tid=1, entry_tid=6088], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:27.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:27.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_37 2024-04-23T21:48:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_37 '..*' 2024-04-23T21:48:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:27.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:27.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:27.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_37.status 2024-04-23T21:48:27.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_37.status 2024-04-23T21:48:27.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:27.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:27.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_37 2024-04-23T21:48:27.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:27.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:27.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:27.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:27.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:27.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6089], [object_number=4, tag_tid=1, entry_tid=6088], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:27.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:27.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:27.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:27.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=6089' '!=' 'object_number=5, tag_tid=1, entry_tid=6089' 2024-04-23T21:48:27.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:27.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:48:29.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_37 2024-04-23T21:48:29.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:29.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:29.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:48:29.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:29.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_37 ']' 2024-04-23T21:48:29.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_37' 2024-04-23T21:48:29.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_37 2024-04-23T21:48:29.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:29.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:29.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:29.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:29.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:29.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:29.639 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:29.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:29.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:29.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:29.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_37 2024-04-23T21:48:29.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:29.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:29.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:29.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:29.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:29.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:29.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:29.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:29.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:29.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:29.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:29.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:29.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:29.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:29.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:29.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:29.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:29.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:29.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:29.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2024-04-23T21:48:29.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:29.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:29.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:29.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_37 2024-04-23T21:48:29.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:29.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:29.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:29.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:29.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:29.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:29.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:29.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:29.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:29.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:29.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:29.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:29.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:29.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:29.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:29.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:29.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:29.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:29.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:29.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:29.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2024-04-23T21:48:29.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:29.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:29.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:29.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_37 2024-04-23T21:48:29.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:29.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:29.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:29.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:29.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:29.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:29.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:29.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:29.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:29.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:29.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:29.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:29.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:29.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:29.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:29.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:29.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:29.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:29.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:29.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:29.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:29.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:29.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:29.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:29.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:29.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:29.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:29.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:29.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:29.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:29.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:29.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:29.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:29.872 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_37 2024-04-23T21:48:29.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:29.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_37 2024-04-23T21:48:29.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:29.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:29.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:29.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_37 '' 2024-04-23T21:48:29.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:29.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:29.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:29.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:29.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_37.status 2024-04-23T21:48:29.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_37.status 2024-04-23T21:48:29.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:29.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:29.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_37 2024-04-23T21:48:29.966 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:30.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:30.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:30.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:30.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:30.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:30.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:30.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:30.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_37 2024-04-23T21:48:30.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:30.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:30.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:30.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_37 '..*' 2024-04-23T21:48:30.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:30.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:30.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:48:30.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:30.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_37.status 2024-04-23T21:48:30.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_37.status 2024-04-23T21:48:30.016 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:30.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:30.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_37 2024-04-23T21:48:30.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:30.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:30.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:30.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:30.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:30.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6087], [object_number=6, tag_tid=1, entry_tid=6086]]], state=connected] 2024-04-23T21:48:30.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.status 2024-04-23T21:48:30.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:30.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_37 snap 2024-04-23T21:48:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:48:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:48:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:48:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:48:30.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:48:31.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_37@snap 2024-04-23T21:48:31.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_37': 2024-04-23T21:48:31.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:48:31.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:48:31.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:48:31.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10b5b6957c25 2024-04-23T21:48:31.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10b5b6957c25 2024-04-23T21:48:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:48:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:48:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:48:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:48:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:37:59 2024 2024-04-23T21:48:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:37:59 2024 2024-04-23T21:48:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:39:16 2024 2024-04-23T21:48:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:48:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10b5b6957c25 2024-04-23T21:48:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:48:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:48:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 72d5e909-e795-4221-ae83-25db4c470a5e 2024-04-23T21:48:31.110 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:48:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_37 snap 2024-04-23T21:48:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:48:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:48:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:48:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.export 2024-04-23T21:48:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_37.export 2024-04-23T21:48:31.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_37.export 2024-04-23T21:48:31.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_37@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.export 2024-04-23T21:48:31.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:48:31.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_37@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_37.export 2024-04-23T21:48:31.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:48:31.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_37.export 2024-04-23T21:48:31.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_37.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_37.export 2024-04-23T21:48:31.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:31.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:48:31.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_38 2024-04-23T21:48:31.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_38 snap 2024-04-23T21:48:31.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:31.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:31.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:48:31.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:48:31.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_38@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:48:33.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_38 2024-04-23T21:48:33.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:33.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:33.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:48:33.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_38 started 2024-04-23T21:48:33.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:33.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:33.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:48:33.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:48:33.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:48:33.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:33.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:48:34.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_38 started 2024-04-23T21:48:34.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:34.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:34.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:48:34.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:48:34.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:48:34.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:48:34.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_38 2024-04-23T21:48:34.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:48:34.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:48:34.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:34.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:34.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:48:34.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:48:34.049 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:34.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:48:34.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:34.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:48:34.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_38 2024-04-23T21:48:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:48:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:48:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:48:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:34.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:34.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:34.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:48:34.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:48:34.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:48:34.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:34.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:48:34.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:48:34.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:48:34.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:34.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:34.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:34.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:48:34.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:48:34.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:48:34.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:48:34.055 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:34.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:34.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:34.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:34.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_38 2024-04-23T21:48:34.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_38'] 2024-04-23T21:48:34.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:48:34.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:34.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:34.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_38 2024-04-23T21:48:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:48:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:48:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:48:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:48:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:48:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:48:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:48:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:48:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:48:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:48:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:48:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:48:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:48:34.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_38 2024-04-23T21:48:34.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:48:34.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:48:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:48:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:48:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:48:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:48:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:48:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_38 2024-04-23T21:48:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:48:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:48:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:48:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_38 2024-04-23T21:48:34.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:48:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:48:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:48:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:48:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:48:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:34.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:48:34.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_38 2024-04-23T21:48:34.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:34.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:34.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:48:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_38 ']' 2024-04-23T21:48:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_38' 2024-04-23T21:48:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_38 2024-04-23T21:48:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:34.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:34.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:34.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_38 2024-04-23T21:48:34.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:34.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:34.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:34.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:34.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:34.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:34.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:34.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_38 2024-04-23T21:48:34.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_38'] 2024-04-23T21:48:34.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:34.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:34.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:34.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_38 2024-04-23T21:48:34.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:34.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:34.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:34.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:34.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:34.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:34.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:34.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:34.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:34.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:34.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:34.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:34.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:34.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:34.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:34.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:34.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:34.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:34.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:34.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:34.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:34.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:34.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:34.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:34.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:34.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:34.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:34.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:34.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:34.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:34.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:34.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:34.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:34.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_38 2024-04-23T21:48:34.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_38 2024-04-23T21:48:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_38 2024-04-23T21:48:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_38 '' 2024-04-23T21:48:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_38 2024-04-23T21:48:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_38.status 2024-04-23T21:48:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_38.status 2024-04-23T21:48:34.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:34.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_38.status 2024-04-23T21:48:34.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_38 2024-04-23T21:48:34.697 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_38.status 2024-04-23T21:48:34.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:34.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:34.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:34.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7099], [object_number=6, tag_tid=1, entry_tid=7098]]], state=connected] 2024-04-23T21:48:34.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7099], [object_number=6, tag_tid=1, entry_tid=7098]]], state=connected] 2024-04-23T21:48:34.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_38.status 2024-04-23T21:48:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:34.745 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_38 2024-04-23T21:48:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_38 2024-04-23T21:48:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_38 '..*' 2024-04-23T21:48:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_38 2024-04-23T21:48:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_38.status 2024-04-23T21:48:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_38.status 2024-04-23T21:48:34.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:34.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_38.status 2024-04-23T21:48:34.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_38 2024-04-23T21:48:34.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_38.status 2024-04-23T21:48:34.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:34.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:34.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:34.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7099], [object_number=6, tag_tid=1, entry_tid=7098]]], state=connected] 2024-04-23T21:48:34.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7099], [object_number=6, tag_tid=1, entry_tid=7098]]], state=connected] 2024-04-23T21:48:34.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_38.status 2024-04-23T21:48:34.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:34.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:34.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:34.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_38 snap 2024-04-23T21:48:34.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:34.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:34.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:48:34.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:48:34.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:48:34.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:48:34.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:48:35.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_38@snap 2024-04-23T21:48:35.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_38': 2024-04-23T21:48:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:48:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:48:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:48:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 116e16d5624d 2024-04-23T21:48:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.116e16d5624d 2024-04-23T21:48:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:48:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:48:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:48:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:48:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:38:07 2024 2024-04-23T21:48:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:38:07 2024 2024-04-23T21:48:35.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:39:08 2024 2024-04-23T21:48:35.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:48:35.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 116e16d5624d 2024-04-23T21:48:35.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:48:35.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:48:35.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 5557bd3b-c922-47cc-a17a-ad2098e15630 2024-04-23T21:48:35.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:48:35.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:35.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_38 snap 2024-04-23T21:48:35.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:48:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:48:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:48:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_38.export 2024-04-23T21:48:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_38.export 2024-04-23T21:48:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_38.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_38.export 2024-04-23T21:48:35.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_38@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_38.export 2024-04-23T21:48:35.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:48:36.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_38@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_38.export 2024-04-23T21:48:36.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:48:36.232 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_38.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_38.export 2024-04-23T21:48:36.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_38.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_38.export 2024-04-23T21:48:36.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:36.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:48:36.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_39 2024-04-23T21:48:36.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_39 snap 2024-04-23T21:48:36.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:36.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:36.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:48:36.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:48:36.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_39@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:48:38.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_39 2024-04-23T21:48:38.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:38.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:38.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:48:38.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_39 started 2024-04-23T21:48:38.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:38.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:38.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:48:38.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:48:38.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:48:38.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:38.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:48:39.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_39 started 2024-04-23T21:48:39.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:39.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:39.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:48:39.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:48:39.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:48:39.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:48:39.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_39 2024-04-23T21:48:39.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:48:39.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:48:39.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:39.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:39.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:48:39.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:48:39.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:39.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:48:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:39.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:48:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_39 2024-04-23T21:48:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:48:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:48:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:48:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:48:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:48:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:48:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:48:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:48:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:48:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:39.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:48:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:48:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:48:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:48:39.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:39.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:39.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:39.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:39.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_39 2024-04-23T21:48:39.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2024-04-23T21:48:39.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:48:39.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_39 2024-04-23T21:48:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:48:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:48:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:39.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:48:39.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:39.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:39.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:39.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:39.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:48:39.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:48:39.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:48:39.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:39.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:48:39.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:48:39.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:48:39.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:39.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:39.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:39.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:48:39.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:48:39.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:48:39.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:48:39.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:39.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:39.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:39.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:39.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_39 2024-04-23T21:48:39.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2024-04-23T21:48:39.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:48:39.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:39.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:39.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_39 2024-04-23T21:48:39.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:48:39.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:39.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:39.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:39.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:39.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:48:39.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:39.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:39.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:39.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:39.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:48:39.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:39.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:39.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:39.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:39.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:48:39.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:48:39.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:48:39.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:39.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:48:39.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:48:39.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:48:39.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:39.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:39.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:39.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:48:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:48:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:48:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:48:39.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:39.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:39.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:39.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:39.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_39 2024-04-23T21:48:39.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:48:39.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:48:39.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:48:39.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:48:39.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:48:39.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:48:39.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:48:39.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:39.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_39 2024-04-23T21:48:39.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:48:39.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:39.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:39.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:48:39.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:48:39.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_39 2024-04-23T21:48:39.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:48:39.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:39.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:39.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:48:39.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:48:39.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:48:39.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:48:39.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:39.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:48:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_39 2024-04-23T21:48:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:48:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_39 ']' 2024-04-23T21:48:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_39' 2024-04-23T21:48:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_39 2024-04-23T21:48:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:39.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:39.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:39.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:39.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:39.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_39 2024-04-23T21:48:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:39.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:39.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:39.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:39.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:39.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:39.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:39.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:39.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2024-04-23T21:48:39.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:39.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_39 2024-04-23T21:48:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:39.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:39.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:39.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:39.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:39.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:39.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:39.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:39.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:39.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:39.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:39.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:39.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:39.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2024-04-23T21:48:39.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:39.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:39.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:39.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_39 2024-04-23T21:48:39.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:39.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:39.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:39.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:39.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:39.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:39.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:39.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:39.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_39 2024-04-23T21:48:39.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:39.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:39.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:39.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_39 '' 2024-04-23T21:48:39.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:39.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:39.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:39.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:39.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_39.status 2024-04-23T21:48:39.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_39.status 2024-04-23T21:48:39.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:39.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:39.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_39 2024-04-23T21:48:39.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:39.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:39.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:39.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:39.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=6266], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:39.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=6266], [object_number=5, tag_tid=1, entry_tid=6265], [object_number=4, tag_tid=1, entry_tid=6264], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:39.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:39.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:39.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_39 2024-04-23T21:48:39.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:39.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:39.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:39.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_39 '..*' 2024-04-23T21:48:39.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:39.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:39.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:39.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:39.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_39.status 2024-04-23T21:48:39.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_39.status 2024-04-23T21:48:39.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:39.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:39.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_39 2024-04-23T21:48:39.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:40.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:40.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:40.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:40.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=6266], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:40.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=6266], [object_number=5, tag_tid=1, entry_tid=6265], [object_number=4, tag_tid=1, entry_tid=6264], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:40.007 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:40.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:40.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:40.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=6266' '!=' '' 2024-04-23T21:48:40.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:48:40.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=6266' = '' 2024-04-23T21:48:40.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:40.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:40.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:48:40.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:40.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:48:40.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:40.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:48:40.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:48:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:48:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:40.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:48:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:48:40.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:40.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=6266 2024-04-23T21:48:40.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 6266 2024-04-23T21:48:40.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:48:40.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:40.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:48:40.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_39 2024-04-23T21:48:40.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:40.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:40.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:48:40.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:40.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_39 ']' 2024-04-23T21:48:40.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_39' 2024-04-23T21:48:40.217 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_39 2024-04-23T21:48:40.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:40.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:40.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:40.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:40.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:40.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:40.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:40.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:40.218 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:40.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:40.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_39 2024-04-23T21:48:40.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:40.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:40.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:40.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:40.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:40.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:40.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:40.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:40.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:40.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:40.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:40.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:40.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:40.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:40.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:40.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:40.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:40.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:40.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:40.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:40.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:40.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:40.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:40.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2024-04-23T21:48:40.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:40.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:40.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:40.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_39 2024-04-23T21:48:40.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:40.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:40.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:40.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:40.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:40.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:40.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:40.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:40.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:40.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:40.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:40.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:40.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:40.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:40.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:40.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:40.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:40.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:40.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:40.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:40.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:40.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:40.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:40.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:40.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:40.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:40.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:40.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:40.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:40.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:40.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:40.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:40.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:40.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:40.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2024-04-23T21:48:40.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:40.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:40.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:40.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_39 2024-04-23T21:48:40.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:40.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:40.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:40.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:40.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:40.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:40.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:40.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:40.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:40.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:40.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:40.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:40.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:40.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:40.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:40.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:40.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:40.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:40.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:40.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:40.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:40.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:40.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:40.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:40.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:40.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:40.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:40.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:40.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:40.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:40.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:40.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:40.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:40.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:40.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:40.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_39 2024-04-23T21:48:40.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:40.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:40.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:40.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_39 '' 2024-04-23T21:48:40.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:40.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:40.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:40.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:40.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_39.status 2024-04-23T21:48:40.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_39.status 2024-04-23T21:48:40.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:40.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:40.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_39 2024-04-23T21:48:40.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:40.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:40.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:40.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:40.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=6266], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:40.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=6266], [object_number=5, tag_tid=1, entry_tid=6265], [object_number=4, tag_tid=1, entry_tid=6264], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:40.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:40.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:40.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_39 2024-04-23T21:48:40.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:40.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:40.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:40.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_39 '..*' 2024-04-23T21:48:40.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:40.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:40.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:40.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:40.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_39.status 2024-04-23T21:48:40.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_39.status 2024-04-23T21:48:40.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:40.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:40.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_39 2024-04-23T21:48:40.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:40.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:40.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:40.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:40.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=6266], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:40.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=6266], [object_number=5, tag_tid=1, entry_tid=6265], [object_number=4, tag_tid=1, entry_tid=6264], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:40.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:40.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:40.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:40.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=6266' '!=' 'object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:40.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:40.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:48:41.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_39 2024-04-23T21:48:41.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:41.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:41.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:48:41.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:41.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_39 ']' 2024-04-23T21:48:41.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_39' 2024-04-23T21:48:41.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_39 2024-04-23T21:48:41.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:41.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:41.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:41.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:41.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:41.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:41.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:41.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:41.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:41.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:41.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_39 2024-04-23T21:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:41.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:41.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:41.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:41.051 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:41.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:41.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2024-04-23T21:48:41.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:41.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_39 2024-04-23T21:48:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:41.135 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:41.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:41.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:41.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:41.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:41.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:41.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2024-04-23T21:48:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:41.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_39 2024-04-23T21:48:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:41.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:41.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:41.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:41.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:41.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:41.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:41.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:41.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:41.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:41.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:41.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:41.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:41.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_39 2024-04-23T21:48:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_39 '' 2024-04-23T21:48:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_39.status 2024-04-23T21:48:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_39.status 2024-04-23T21:48:41.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_39 2024-04-23T21:48:41.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:41.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:41.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:41.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:41.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=6266], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:41.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=6266], [object_number=5, tag_tid=1, entry_tid=6265], [object_number=4, tag_tid=1, entry_tid=6264], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:41.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:41.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:41.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_39 2024-04-23T21:48:41.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:41.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:41.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:41.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_39 '..*' 2024-04-23T21:48:41.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:41.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:41.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:41.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:41.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_39.status 2024-04-23T21:48:41.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_39.status 2024-04-23T21:48:41.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:41.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:41.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_39 2024-04-23T21:48:41.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:41.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:41.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:41.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:41.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=6266], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:41.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=6266], [object_number=5, tag_tid=1, entry_tid=6265], [object_number=4, tag_tid=1, entry_tid=6264], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:41.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:41.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:41.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:41.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=6266' '!=' 'object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:41.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:41.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:48:42.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_39 2024-04-23T21:48:42.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:42.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:42.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:48:42.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:42.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_39 ']' 2024-04-23T21:48:42.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_39' 2024-04-23T21:48:42.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_39 2024-04-23T21:48:42.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:42.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:42.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:42.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:42.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:42.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:42.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:42.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:42.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:42.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:42.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_39 2024-04-23T21:48:42.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:42.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:42.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:42.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:42.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:42.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:42.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:42.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:42.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:42.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:42.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:42.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:42.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:42.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:42.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:42.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:42.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:42.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:42.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:42.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:42.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2024-04-23T21:48:42.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:42.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:42.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:42.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_39 2024-04-23T21:48:42.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:42.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:42.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:42.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:42.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:42.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:42.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:42.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:42.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:42.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:42.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:42.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:42.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:42.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:42.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:42.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:42.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:42.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:42.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:42.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:42.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2024-04-23T21:48:42.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:42.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:42.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:42.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_39 2024-04-23T21:48:42.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:42.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:42.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:42.448 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:42.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:42.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:42.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:42.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:42.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:42.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:42.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:42.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_39 2024-04-23T21:48:42.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:42.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:42.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:42.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_39 '' 2024-04-23T21:48:42.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:42.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:42.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:42.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:42.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_39.status 2024-04-23T21:48:42.556 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_39.status 2024-04-23T21:48:42.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:42.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:42.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_39 2024-04-23T21:48:42.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:42.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:42.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:42.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:42.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=6266], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:42.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=6266], [object_number=5, tag_tid=1, entry_tid=6265], [object_number=4, tag_tid=1, entry_tid=6264], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:42.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:42.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:42.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_39 2024-04-23T21:48:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_39 '..*' 2024-04-23T21:48:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_39.status 2024-04-23T21:48:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_39.status 2024-04-23T21:48:42.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:42.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:42.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_39 2024-04-23T21:48:42.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:42.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:42.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:42.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:42.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=6266], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:42.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=6266], [object_number=5, tag_tid=1, entry_tid=6265], [object_number=4, tag_tid=1, entry_tid=6264], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:42.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:42.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:42.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:42.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=6266' '!=' 'object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:42.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:42.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:48:44.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_39 2024-04-23T21:48:44.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:44.253 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:44.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:48:44.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:44.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_39 ']' 2024-04-23T21:48:44.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_39' 2024-04-23T21:48:44.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_39 2024-04-23T21:48:44.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:44.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:44.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:44.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:44.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:44.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:44.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:44.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:44.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:44.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:44.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_39 2024-04-23T21:48:44.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:44.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:44.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:44.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:44.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:44.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:44.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:44.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:44.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:44.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:44.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:44.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:44.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:44.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:44.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:44.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:44.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:44.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:44.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:44.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:44.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:44.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:44.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:44.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:44.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:44.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:44.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:44.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:44.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:44.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:44.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:44.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:44.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:44.280 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:44.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2024-04-23T21:48:44.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:44.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:44.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:44.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_39 2024-04-23T21:48:44.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:44.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:44.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:44.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:44.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:44.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:44.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:44.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:44.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:44.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:44.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:44.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:44.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:44.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:44.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:44.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:44.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:44.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:44.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:44.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:44.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2024-04-23T21:48:44.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:44.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:44.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:44.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_39 2024-04-23T21:48:44.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:44.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:44.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:44.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:44.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:44.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:44.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:44.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:44.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:44.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:44.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_39 2024-04-23T21:48:44.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:44.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:44.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:44.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_39 '' 2024-04-23T21:48:44.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:44.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:44.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:44.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:44.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_39.status 2024-04-23T21:48:44.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_39.status 2024-04-23T21:48:44.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:44.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:44.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_39 2024-04-23T21:48:44.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:44.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:44.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:44.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:44.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=6266], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:44.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=6266], [object_number=5, tag_tid=1, entry_tid=6265], [object_number=4, tag_tid=1, entry_tid=6264], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:44.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:44.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_39 2024-04-23T21:48:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_39 '..*' 2024-04-23T21:48:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:44.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_39.status 2024-04-23T21:48:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_39.status 2024-04-23T21:48:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_39 2024-04-23T21:48:44.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:44.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:44.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:44.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:44.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=6266], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:44.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=6266], [object_number=5, tag_tid=1, entry_tid=6265], [object_number=4, tag_tid=1, entry_tid=6264], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:44.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:44.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:44.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:44.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=6266' '!=' 'object_number=6, tag_tid=1, entry_tid=6266' 2024-04-23T21:48:44.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:44.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:48:46.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_39 2024-04-23T21:48:46.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:46.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:46.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:48:46.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:46.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_39 ']' 2024-04-23T21:48:46.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_39' 2024-04-23T21:48:46.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_39 2024-04-23T21:48:46.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:46.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:46.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:46.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:46.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:46.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:46.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_39 2024-04-23T21:48:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:46.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:46.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:46.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:46.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:46.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2024-04-23T21:48:46.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:46.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:46.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:46.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_39 2024-04-23T21:48:46.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:46.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:46.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:46.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:46.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:46.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:46.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:46.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:46.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:46.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:46.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:46.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:46.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:46.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:46.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:46.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:46.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:46.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:46.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:46.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:46.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:46.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2024-04-23T21:48:46.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:46.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:46.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:46.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_39 2024-04-23T21:48:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:46.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:46.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:46.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:46.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:46.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:46.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:46.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_39 2024-04-23T21:48:47.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:47.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_39 2024-04-23T21:48:47.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:47.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:47.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:47.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_39 '' 2024-04-23T21:48:47.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:47.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:47.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:47.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:47.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_39.status 2024-04-23T21:48:47.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_39.status 2024-04-23T21:48:47.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:47.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:47.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_39 2024-04-23T21:48:47.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:47.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:47.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:47.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:47.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=6266], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:47.048 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=6266], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:47.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_39 2024-04-23T21:48:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_39 '..*' 2024-04-23T21:48:47.052 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:47.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:47.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:48:47.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:47.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_39.status 2024-04-23T21:48:47.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_39.status 2024-04-23T21:48:47.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:47.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:47.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_39 2024-04-23T21:48:47.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:47.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:48:47.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:48:47.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:47.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=6266], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:47.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=6266], [object_number=7, tag_tid=1, entry_tid=6263]]], state=connected] 2024-04-23T21:48:47.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.status 2024-04-23T21:48:47.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:47.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:48:47.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:47.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_39 snap 2024-04-23T21:48:47.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:47.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:47.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:48:47.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:48:47.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:48:47.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:48:47.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:48:48.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_39@snap 2024-04-23T21:48:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_39': 2024-04-23T21:48:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:48:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:48:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:48:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 11b83447cdf6 2024-04-23T21:48:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.11b83447cdf6 2024-04-23T21:48:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:48:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:48:48.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:48:48.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:48:48.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:38:16 2024 2024-04-23T21:48:48.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:38:16 2024 2024-04-23T21:48:48.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:38:16 2024 2024-04-23T21:48:48.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:48:48.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 11b83447cdf6 2024-04-23T21:48:48.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:48:48.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:48:48.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: ad5d9133-f827-4484-a69d-85cb1ef726c4 2024-04-23T21:48:48.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:48:48.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:48.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_39 snap 2024-04-23T21:48:48.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:48.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:48:48.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:48:48.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:48:48.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.export 2024-04-23T21:48:48.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_39.export 2024-04-23T21:48:48.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_39.export 2024-04-23T21:48:48.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_39@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.export 2024-04-23T21:48:48.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:48:48.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_39@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_39.export 2024-04-23T21:48:48.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:48:48.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_39.export 2024-04-23T21:48:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_39.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_39.export 2024-04-23T21:48:48.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:48.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:48:48.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_40 2024-04-23T21:48:48.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_40 snap 2024-04-23T21:48:48.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:48.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:48.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:48:48.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:48:48.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_40@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:48:50.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_40 2024-04-23T21:48:50.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:50.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:50.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:48:50.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_40 started 2024-04-23T21:48:50.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:50.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:50.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:48:50.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:48:50.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:48:50.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:50.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:48:51.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_40 started 2024-04-23T21:48:51.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:51.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:51.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:48:51.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:48:51.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:48:51.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:48:51.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_40 2024-04-23T21:48:51.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:48:51.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:51.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:48:51.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:51.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:48:51.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:48:51.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:51.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:48:51.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:51.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:48:51.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:51.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_40 2024-04-23T21:48:51.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:48:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:48:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:48:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:51.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:48:51.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:48:51.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:48:51.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:51.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:48:51.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:48:51.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:48:51.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:51.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:51.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:51.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:48:51.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:48:51.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:48:51.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:48:51.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:51.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:51.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:51.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:51.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_40 2024-04-23T21:48:51.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_40'] 2024-04-23T21:48:51.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:48:51.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:51.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:51.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_40 2024-04-23T21:48:51.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:48:51.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:51.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:51.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:51.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:51.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:48:51.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:51.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:48:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:48:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:48:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:48:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:48:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:48:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:48:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:51.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:51.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:51.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:48:51.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:48:51.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:48:51.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:48:51.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:51.224 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_40 2024-04-23T21:48:51.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_40'] 2024-04-23T21:48:51.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:48:51.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:51.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:51.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_40 2024-04-23T21:48:51.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:48:51.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:48:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:48:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:48:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:48:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:48:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:48:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:48:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:48:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:48:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:48:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:48:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:48:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:48:51.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:48:51.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:51.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:51.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:51.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:51.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_40 2024-04-23T21:48:51.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:48:51.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:48:51.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:48:51.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:48:51.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:48:51.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:48:51.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:48:51.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:51.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_40 2024-04-23T21:48:51.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:48:51.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:51.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:51.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:48:51.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:48:51.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_40 2024-04-23T21:48:51.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:48:51.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:48:51.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:51.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:48:51.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:48:51.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:48:51.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:48:51.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:51.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:48:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_40 2024-04-23T21:48:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:48:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_40 ']' 2024-04-23T21:48:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_40' 2024-04-23T21:48:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_40 2024-04-23T21:48:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:51.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:51.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:51.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:51.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:51.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:51.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:51.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:51.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:51.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_40 2024-04-23T21:48:51.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:51.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:51.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:51.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:51.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:51.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:51.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:51.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:51.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:51.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:51.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:51.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:51.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:51.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:51.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:51.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:51.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:51.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:51.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:51.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:51.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:51.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:51.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:51.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:51.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:51.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:51.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:51.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:51.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:51.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:51.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:51.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:51.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:51.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:51.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_40'] 2024-04-23T21:48:51.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:51.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:51.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:51.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_40 2024-04-23T21:48:51.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:51.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:51.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:51.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:51.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:51.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:51.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:51.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:51.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:51.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:51.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:51.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:51.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:51.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:51.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:51.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:51.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:51.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:51.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:51.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:51.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:51.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:51.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:51.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:51.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:51.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:51.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:51.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:51.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:51.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:51.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:51.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:51.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:51.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_40'] 2024-04-23T21:48:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:51.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:51.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:51.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_40 2024-04-23T21:48:51.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:51.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:51.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:51.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:51.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:51.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:51.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:51.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:51.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:51.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:51.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:51.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:51.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:51.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:51.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:51.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:51.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:51.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:51.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:51.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:51.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:51.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_40 2024-04-23T21:48:51.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:51.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:51.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:51.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_40 '' 2024-04-23T21:48:51.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:51.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:51.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:51.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:51.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_40.status 2024-04-23T21:48:51.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_40.status 2024-04-23T21:48:51.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:51.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:51.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_40 2024-04-23T21:48:51.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:51.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:51.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:51.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:51.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11546], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:51.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11546], [object_number=9, tag_tid=1, entry_tid=11545], [object_number=8, tag_tid=1, entry_tid=11544], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:51.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:51.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:48:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_40 2024-04-23T21:48:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_40 '..*' 2024-04-23T21:48:51.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:51.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:51.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:51.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:51.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_40.status 2024-04-23T21:48:51.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_40.status 2024-04-23T21:48:51.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:51.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:51.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_40 2024-04-23T21:48:51.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:52.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:52.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:52.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:52.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11546], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:52.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11546], [object_number=9, tag_tid=1, entry_tid=11545], [object_number=8, tag_tid=1, entry_tid=11544], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:52.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:52.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:52.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:52.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=11546' '!=' '' 2024-04-23T21:48:52.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:48:52.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=11546' = '' 2024-04-23T21:48:52.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:52.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:48:52.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:48:52.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:48:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:52.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:48:52.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:52.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:48:52.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:48:52.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:48:52.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:52.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:48:52.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:52.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:48:52.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:48:52.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=11546 2024-04-23T21:48:52.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 11546 2024-04-23T21:48:52.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:48:52.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:52.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:48:52.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_40 2024-04-23T21:48:52.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:52.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:52.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:48:52.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:52.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_40 ']' 2024-04-23T21:48:52.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_40' 2024-04-23T21:48:52.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_40 2024-04-23T21:48:52.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:52.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:52.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:52.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:52.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:52.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:52.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:52.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:52.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:52.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:52.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_40 2024-04-23T21:48:52.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:52.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:52.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:52.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:52.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:52.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:52.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:52.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:52.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:52.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:52.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:52.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:52.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:52.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:52.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:52.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:52.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:52.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:52.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:52.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:52.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:52.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:52.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:52.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:52.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:52.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:52.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:52.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:52.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:52.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:52.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:52.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:52.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:52.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:52.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_40'] 2024-04-23T21:48:52.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:52.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:52.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_40 2024-04-23T21:48:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:52.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:52.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:52.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:52.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:52.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:52.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:52.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:52.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:52.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:52.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_40'] 2024-04-23T21:48:52.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:52.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:52.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:52.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_40 2024-04-23T21:48:52.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:52.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:52.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:52.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:52.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:52.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:52.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:52.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:52.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:52.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:52.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:52.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:52.468 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:52.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:52.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:52.559 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:52.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_40 2024-04-23T21:48:52.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:52.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:52.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:52.560 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_40 '' 2024-04-23T21:48:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_40.status 2024-04-23T21:48:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_40.status 2024-04-23T21:48:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:52.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:52.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_40 2024-04-23T21:48:52.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:52.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:52.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:52.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:52.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11546], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:52.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11546], [object_number=9, tag_tid=1, entry_tid=11545], [object_number=8, tag_tid=1, entry_tid=11544], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:52.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:52.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:48:52.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_40 2024-04-23T21:48:52.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:52.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:52.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:52.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_40 '..*' 2024-04-23T21:48:52.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:52.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:52.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:52.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:52.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_40.status 2024-04-23T21:48:52.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_40.status 2024-04-23T21:48:52.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:52.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:52.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_40 2024-04-23T21:48:52.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:52.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:52.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:52.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:52.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11546], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:52.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11546], [object_number=9, tag_tid=1, entry_tid=11545], [object_number=8, tag_tid=1, entry_tid=11544], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:52.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:52.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:52.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:52.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=11546' '!=' 'object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:52.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:52.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:48:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_40 2024-04-23T21:48:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:48:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_40 ']' 2024-04-23T21:48:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_40' 2024-04-23T21:48:53.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_40 2024-04-23T21:48:53.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:53.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:53.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:53.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:53.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:53.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:53.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:53.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:53.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:53.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_40 2024-04-23T21:48:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:53.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:53.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:53.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:53.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:53.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:53.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:53.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:53.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:53.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:53.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:53.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:53.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:53.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:53.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:53.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:53.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:53.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:53.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:53.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:53.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:53.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:53.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_40'] 2024-04-23T21:48:53.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:53.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:53.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:53.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_40 2024-04-23T21:48:53.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:53.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:53.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:53.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:53.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:53.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:53.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:53.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:53.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:53.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:53.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:53.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:53.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:53.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:53.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:53.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:53.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:53.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:53.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:53.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_40'] 2024-04-23T21:48:53.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:53.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:53.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:53.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_40 2024-04-23T21:48:53.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:53.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:53.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:53.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:53.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:53.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:53.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:53.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:53.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:53.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:53.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:53.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:53.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:53.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:53.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:53.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:53.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:53.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:53.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:53.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:53.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:53.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:53.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:53.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:53.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:53.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:53.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:53.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:53.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:53.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:53.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:53.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:53.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:53.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:53.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:53.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_40 2024-04-23T21:48:53.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:53.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:53.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:53.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_40 '' 2024-04-23T21:48:53.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:53.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:53.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:53.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:53.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_40.status 2024-04-23T21:48:53.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_40.status 2024-04-23T21:48:53.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:53.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:53.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_40 2024-04-23T21:48:53.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:53.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:53.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:53.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:53.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11546], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:53.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11546], [object_number=9, tag_tid=1, entry_tid=11545], [object_number=8, tag_tid=1, entry_tid=11544], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:53.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:53.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:48:53.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_40 2024-04-23T21:48:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_40 '..*' 2024-04-23T21:48:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_40.status 2024-04-23T21:48:53.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_40.status 2024-04-23T21:48:53.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:53.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:53.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_40 2024-04-23T21:48:53.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:53.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:53.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:53.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:53.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11546], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:53.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11546], [object_number=9, tag_tid=1, entry_tid=11545], [object_number=8, tag_tid=1, entry_tid=11544], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:53.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:53.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:53.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:53.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=11546' '!=' 'object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:53.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:53.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:48:54.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_40 2024-04-23T21:48:54.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:54.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:54.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:48:54.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:54.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_40 ']' 2024-04-23T21:48:54.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_40' 2024-04-23T21:48:54.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_40 2024-04-23T21:48:54.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:54.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:54.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:54.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:54.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:54.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:54.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:54.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:54.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:54.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:54.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_40 2024-04-23T21:48:54.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:54.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:54.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:54.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:54.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:54.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:54.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:54.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:54.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:54.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:54.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:54.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:54.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:54.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:54.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:54.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_40'] 2024-04-23T21:48:54.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:54.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:54.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:54.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_40 2024-04-23T21:48:54.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:54.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:54.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:54.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:54.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:54.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:54.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:54.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:54.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:54.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:54.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:54.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:54.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:54.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:54.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:54.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:54.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:54.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:54.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:54.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:54.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:54.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:54.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:54.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:54.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:54.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:54.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:54.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:54.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:54.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:54.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:54.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:54.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:54.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:54.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_40'] 2024-04-23T21:48:54.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:54.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:54.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:54.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_40 2024-04-23T21:48:54.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:54.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:54.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:54.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:54.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:54.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:54.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:54.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:54.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:54.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:54.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:54.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:54.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:54.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:54.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:54.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:54.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:54.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:54.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:54.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:54.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:54.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:54.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:54.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:54.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:54.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:54.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:54.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:54.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:54.477 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:54.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:54.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:54.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:54.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:54.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_40 2024-04-23T21:48:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_40 '' 2024-04-23T21:48:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_40.status 2024-04-23T21:48:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_40.status 2024-04-23T21:48:54.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:54.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:54.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:54.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_40 2024-04-23T21:48:54.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:54.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:54.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:54.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11546], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:54.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11546], [object_number=9, tag_tid=1, entry_tid=11545], [object_number=8, tag_tid=1, entry_tid=11544], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:54.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:54.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:48:54.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_40 2024-04-23T21:48:54.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:54.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:54.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:54.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_40 '..*' 2024-04-23T21:48:54.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:54.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:54.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:54.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:54.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_40.status 2024-04-23T21:48:54.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_40.status 2024-04-23T21:48:54.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:54.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:54.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_40 2024-04-23T21:48:54.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:54.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:54.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:54.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:54.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11546], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:54.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11546], [object_number=9, tag_tid=1, entry_tid=11545], [object_number=8, tag_tid=1, entry_tid=11544], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:54.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:54.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:54.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:54.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=11546' '!=' 'object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:54.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:54.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:48:56.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_40 2024-04-23T21:48:56.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:56.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:56.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:48:56.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:56.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_40 ']' 2024-04-23T21:48:56.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_40' 2024-04-23T21:48:56.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_40 2024-04-23T21:48:56.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:56.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:56.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:56.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:56.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:56.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:56.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:56.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:56.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:56.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:56.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_40 2024-04-23T21:48:56.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:56.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:56.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:56.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:56.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:56.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:56.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:56.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:56.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:56.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:56.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:56.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:56.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:56.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:56.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:56.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:56.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:56.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:56.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:56.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:56.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:56.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:56.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:56.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:56.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:56.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:56.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:56.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:56.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:56.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:56.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:56.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_40'] 2024-04-23T21:48:56.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:56.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:56.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:56.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_40 2024-04-23T21:48:56.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:56.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:56.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:56.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:56.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:56.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:56.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:56.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:56.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_40'] 2024-04-23T21:48:56.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:56.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:56.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:56.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_40 2024-04-23T21:48:56.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:56.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:56.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:56.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:56.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:56.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:56.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:56.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:56.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:56.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:56.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:56.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:56.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:56.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:56.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:56.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:56.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:56.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:56.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:56.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:56.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:56.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:56.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:56.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:56.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:56.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:56.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:56.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:56.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:56.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:56.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:56.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:56.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:56.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:56.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:56.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_40 2024-04-23T21:48:56.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:56.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:56.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:56.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_40 '' 2024-04-23T21:48:56.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:56.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:56.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:56.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:56.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_40.status 2024-04-23T21:48:56.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_40.status 2024-04-23T21:48:56.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:56.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:56.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_40 2024-04-23T21:48:56.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:56.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11546], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:56.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11546], [object_number=9, tag_tid=1, entry_tid=11545], [object_number=8, tag_tid=1, entry_tid=11544], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:56.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:56.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:48:56.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_40 2024-04-23T21:48:56.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:56.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:56.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:56.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_40 '..*' 2024-04-23T21:48:56.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:56.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:56.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:56.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:56.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_40.status 2024-04-23T21:48:56.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_40.status 2024-04-23T21:48:56.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:56.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:56.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_40 2024-04-23T21:48:56.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:56.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:56.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:56.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:56.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11546], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:56.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11546], [object_number=9, tag_tid=1, entry_tid=11545], [object_number=8, tag_tid=1, entry_tid=11544], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:56.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:56.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:56.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:56.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=11546' '!=' 'object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:56.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:56.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:48:58.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_40 2024-04-23T21:48:58.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:58.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:48:58.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:48:58.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:48:58.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_40 ']' 2024-04-23T21:48:58.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_40' 2024-04-23T21:48:58.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_40 2024-04-23T21:48:58.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:48:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:48:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:48:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:48:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:48:58.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:48:58.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:48:58.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:48:58.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:58.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_40 2024-04-23T21:48:58.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:48:58.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:58.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:58.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:48:58.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:48:58.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:58.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:58.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:58.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:48:58.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:58.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:58.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:58.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:48:58.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:58.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:48:58.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:48:58.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:58.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:58.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:48:58.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:48:58.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:58.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:58.703 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:58.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:48:58.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:48:58.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:58.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:58.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:48:58.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:48:58.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:48:58.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:58.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:58.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:48:58.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:58.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_40'] 2024-04-23T21:48:58.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:58.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:58.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:58.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_40 2024-04-23T21:48:58.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:48:58.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:58.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:58.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:48:58.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:48:58.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:58.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:58.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:58.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:48:58.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:58.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:58.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:58.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:48:58.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:58.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:48:58.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:48:58.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:58.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:58.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:48:58.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:48:58.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:58.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:58.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:58.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:48:58.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:48:58.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:58.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:58.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:48:58.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:48:58.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:48:58.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:58.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:58.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:48:58.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:58.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_40'] 2024-04-23T21:48:58.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:48:58.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:48:58.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:48:58.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_40 2024-04-23T21:48:58.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:48:58.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:48:58.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:48:58.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:48:58.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:48:58.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:48:58.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:48:58.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:48:58.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:48:58.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:58.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:48:58.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:58.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:48:58.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:58.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:48:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:48:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:48:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:48:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:48:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:48:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:48:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:48:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:48:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:48:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:48:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:48:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:48:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:48:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:48:58.918 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:48:58.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:58.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:58.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:48:58.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_40 2024-04-23T21:48:59.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:48:59.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_40 2024-04-23T21:48:59.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:59.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:59.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:59.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_40 '' 2024-04-23T21:48:59.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:59.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:59.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:59.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:48:59.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_40.status 2024-04-23T21:48:59.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_40.status 2024-04-23T21:48:59.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:59.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:59.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_40 2024-04-23T21:48:59.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:59.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:59.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:59.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:59.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11546], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:59.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11546], [object_number=9, tag_tid=1, entry_tid=11545], [object_number=8, tag_tid=1, entry_tid=11544], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:59.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:59.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:48:59.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_40 2024-04-23T21:48:59.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:59.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:59.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:59.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_40 '..*' 2024-04-23T21:48:59.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:48:59.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:48:59.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:48:59.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:48:59.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_40.status 2024-04-23T21:48:59.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_40.status 2024-04-23T21:48:59.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:48:59.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:59.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_40 2024-04-23T21:48:59.074 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:59.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:48:59.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:48:59.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:48:59.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11546], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:59.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=10, tag_tid=1, entry_tid=11546], [object_number=9, tag_tid=1, entry_tid=11545], [object_number=8, tag_tid=1, entry_tid=11544], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:48:59.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:48:59.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:59.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:59.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=10, tag_tid=1, entry_tid=11546' '!=' 'object_number=10, tag_tid=1, entry_tid=11546' 2024-04-23T21:48:59.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:48:59.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:49:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_40 2024-04-23T21:49:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:49:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:01.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_40 ']' 2024-04-23T21:49:01.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_40' 2024-04-23T21:49:01.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_40 2024-04-23T21:49:01.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:01.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:01.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:01.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:01.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:01.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:01.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:01.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:01.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:01.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:01.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_40 2024-04-23T21:49:01.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:01.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:01.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:01.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:01.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:01.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:01.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:01.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:01.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:01.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:01.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:01.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:01.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:01.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:01.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:01.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:01.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:01.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:01.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_40 2024-04-23T21:49:01.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_40'] 2024-04-23T21:49:01.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:01.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:01.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:01.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_40 2024-04-23T21:49:01.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:01.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:01.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:01.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:01.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:01.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:01.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:01.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:01.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:01.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:01.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:01.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:01.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:01.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:01.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:01.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:01.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:01.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:01.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:01.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:01.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:01.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:01.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_40 2024-04-23T21:49:01.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_40'] 2024-04-23T21:49:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_40 2024-04-23T21:49:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:49:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:49:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:01.342 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:49:01.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:01.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:01.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:01.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:49:01.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:01.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:01.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:49:01.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:01.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:01.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:01.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:01.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:01.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:01.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:01.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:01.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:01.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:01.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:01.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:01.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:01.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:01.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:01.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:01.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:01.360 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_40 2024-04-23T21:49:01.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:01.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_40 2024-04-23T21:49:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:49:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_40 '' 2024-04-23T21:49:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:49:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_40.status 2024-04-23T21:49:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_40.status 2024-04-23T21:49:01.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:01.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:49:01.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_40 2024-04-23T21:49:01.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:49:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:49:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:49:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11546], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:49:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11546], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:49:01.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:49:01.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:49:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_40 2024-04-23T21:49:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:49:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_40 '..*' 2024-04-23T21:49:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:49:01.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:01.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_40.status 2024-04-23T21:49:01.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_40.status 2024-04-23T21:49:01.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:01.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:49:01.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_40 2024-04-23T21:49:01.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:49:01.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:49:01.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:49:01.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:01.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11546], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:49:01.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=10, tag_tid=1, entry_tid=11546], [object_number=11, tag_tid=1, entry_tid=11543]]], state=connected] 2024-04-23T21:49:01.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.status 2024-04-23T21:49:01.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:49:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:49:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_40 snap 2024-04-23T21:49:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:49:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:49:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:49:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:49:02.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_40@snap 2024-04-23T21:49:02.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_40': 2024-04-23T21:49:02.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:49:02.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:49:02.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:49:02.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 121d115ea2b6 2024-04-23T21:49:02.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.121d115ea2b6 2024-04-23T21:49:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:49:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:49:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:49:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:49:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:38:25 2024 2024-04-23T21:49:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:38:25 2024 2024-04-23T21:49:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:39:26 2024 2024-04-23T21:49:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:49:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 121d115ea2b6 2024-04-23T21:49:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:49:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:49:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 0fed5904-a80e-43ec-9ec8-223012b280e6 2024-04-23T21:49:02.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:49:02.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:02.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_40 snap 2024-04-23T21:49:02.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:02.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:49:02.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:02.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:49:02.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.export 2024-04-23T21:49:02.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_40.export 2024-04-23T21:49:02.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_40.export 2024-04-23T21:49:02.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_40@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.export 2024-04-23T21:49:02.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:49:02.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_40@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_40.export 2024-04-23T21:49:02.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:49:03.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_40.export 2024-04-23T21:49:03.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_40.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_40.export 2024-04-23T21:49:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:49:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_41 2024-04-23T21:49:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_41 snap 2024-04-23T21:49:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:49:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:03.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:49:03.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:03.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_41@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:49:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_41 2024-04-23T21:49:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:49:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_41 started 2024-04-23T21:49:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:06.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:49:06.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:49:06.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:49:06.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:06.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:49:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_41 started 2024-04-23T21:49:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:07.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:49:07.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:49:07.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:49:07.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:49:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_41 2024-04-23T21:49:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:49:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:49:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:49:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:49:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:49:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:07.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:49:07.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:07.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_41 2024-04-23T21:49:07.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:49:07.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:49:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:49:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:49:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:49:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:49:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:49:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:49:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:49:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:49:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:49:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:49:07.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:49:07.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:07.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:07.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:07.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:07.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_41 2024-04-23T21:49:07.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_41'] 2024-04-23T21:49:07.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:49:07.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:07.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:07.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_41 2024-04-23T21:49:07.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:49:07.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:49:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:49:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:07.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:49:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:49:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:49:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:49:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:49:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:49:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:49:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:49:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:49:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:49:07.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:07.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:07.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:07.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:07.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_41 2024-04-23T21:49:07.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:49:07.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:49:07.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:49:07.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:49:07.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:49:07.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:49:07.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:49:07.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:07.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_41 2024-04-23T21:49:07.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:49:07.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:49:07.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:07.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:49:07.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:49:07.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_41 2024-04-23T21:49:07.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:49:07.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:49:07.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:49:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:49:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:49:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:49:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:07.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:49:07.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_41 2024-04-23T21:49:07.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:07.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:07.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:49:07.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:07.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_41 ']' 2024-04-23T21:49:07.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_41' 2024-04-23T21:49:07.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_41 2024-04-23T21:49:07.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:07.572 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:07.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:07.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:07.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:07.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:07.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:07.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:07.573 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:07.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:07.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_41 2024-04-23T21:49:07.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:07.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:07.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:07.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:07.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:07.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:07.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:07.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:07.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:07.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:07.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:07.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:07.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_41 2024-04-23T21:49:07.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_41'] 2024-04-23T21:49:07.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:07.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:07.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:07.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_41 2024-04-23T21:49:07.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:07.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:07.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:07.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:07.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:07.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:07.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:07.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:07.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:07.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:07.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:07.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:07.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:07.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:07.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:07.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:07.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:07.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:07.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:07.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:07.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:07.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:07.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:07.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:07.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:07.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:07.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:07.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:07.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:07.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:07.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:07.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:07.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:07.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_41 2024-04-23T21:49:07.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_41 2024-04-23T21:49:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_41 '' 2024-04-23T21:49:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_41.status 2024-04-23T21:49:07.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_41.status 2024-04-23T21:49:07.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:07.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:07.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_41 2024-04-23T21:49:07.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:07.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:07.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:07.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:07.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=5614], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:07.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=5614], [object_number=5, tag_tid=1, entry_tid=5613], [object_number=4, tag_tid=1, entry_tid=5612], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:07.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:07.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:07.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_41 2024-04-23T21:49:07.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:07.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:07.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:07.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_41 '..*' 2024-04-23T21:49:07.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:07.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:07.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:07.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:07.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_41.status 2024-04-23T21:49:07.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_41.status 2024-04-23T21:49:07.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:07.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:07.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_41 2024-04-23T21:49:07.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:07.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:07.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:07.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:07.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=5614], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:07.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=5614], [object_number=5, tag_tid=1, entry_tid=5613], [object_number=4, tag_tid=1, entry_tid=5612], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:07.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:07.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:07.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:07.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=5614' '!=' '' 2024-04-23T21:49:07.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:49:07.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=5614' = '' 2024-04-23T21:49:07.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:07.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:07.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:49:07.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:07.886 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:49:07.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:07.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:49:07.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:07.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:49:07.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:07.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:49:07.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:07.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:49:07.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:07.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:49:07.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:07.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=5614 2024-04-23T21:49:07.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 5614 2024-04-23T21:49:07.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:49:07.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:07.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:49:08.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_41 2024-04-23T21:49:08.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:08.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:08.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:49:08.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:08.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_41 ']' 2024-04-23T21:49:08.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_41' 2024-04-23T21:49:08.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_41 2024-04-23T21:49:08.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:08.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:08.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:08.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:08.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:08.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:08.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:08.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:08.094 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:08.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:08.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_41 2024-04-23T21:49:08.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:08.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:08.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:08.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:08.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:08.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:08.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:08.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_41 2024-04-23T21:49:08.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_41'] 2024-04-23T21:49:08.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:08.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:08.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:08.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_41 2024-04-23T21:49:08.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:08.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:08.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:08.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:08.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:08.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:08.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:08.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:08.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:08.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:08.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:08.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:08.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:08.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:08.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:08.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:08.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:08.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:08.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:08.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_41 2024-04-23T21:49:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_41 2024-04-23T21:49:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_41 '' 2024-04-23T21:49:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_41.status 2024-04-23T21:49:08.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_41.status 2024-04-23T21:49:08.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:08.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:08.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_41 2024-04-23T21:49:08.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:08.343 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:08.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:08.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:08.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=5614], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:08.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=5614], [object_number=5, tag_tid=1, entry_tid=5613], [object_number=4, tag_tid=1, entry_tid=5612], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:08.346 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_41 2024-04-23T21:49:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_41 '..*' 2024-04-23T21:49:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:08.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:08.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:08.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_41.status 2024-04-23T21:49:08.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_41.status 2024-04-23T21:49:08.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:08.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:08.350 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_41 2024-04-23T21:49:08.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:08.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:08.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:08.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:08.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=5614], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:08.392 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=5614], [object_number=5, tag_tid=1, entry_tid=5613], [object_number=4, tag_tid=1, entry_tid=5612], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:08.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:08.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:08.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:08.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=5614' '!=' 'object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:08.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:08.396 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:49:08.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_41 2024-04-23T21:49:08.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:49:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_41 ']' 2024-04-23T21:49:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_41' 2024-04-23T21:49:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_41 2024-04-23T21:49:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:08.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:08.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_41 2024-04-23T21:49:08.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:08.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:08.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:08.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:08.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:08.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:08.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:08.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:08.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:08.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:08.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:08.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:08.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:08.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:08.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:08.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:08.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:08.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:08.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:08.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:08.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:08.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:08.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:08.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:08.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:08.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:08.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:08.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:08.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:08.805 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:08.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:08.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:08.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:08.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_41 2024-04-23T21:49:08.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_41'] 2024-04-23T21:49:08.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:08.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:08.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:08.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_41 2024-04-23T21:49:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:08.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:08.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:08.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:08.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:08.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:08.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:08.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_41 2024-04-23T21:49:09.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:09.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_41 2024-04-23T21:49:09.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:09.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:09.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:09.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_41 '' 2024-04-23T21:49:09.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:09.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:09.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:09.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:09.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_41.status 2024-04-23T21:49:09.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_41.status 2024-04-23T21:49:09.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:09.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:09.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_41 2024-04-23T21:49:09.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:09.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:09.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:09.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:09.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=5614], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:09.053 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=5614], [object_number=5, tag_tid=1, entry_tid=5613], [object_number=4, tag_tid=1, entry_tid=5612], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:09.056 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:09.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:09.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_41 2024-04-23T21:49:09.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_41 '..*' 2024-04-23T21:49:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_41.status 2024-04-23T21:49:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_41.status 2024-04-23T21:49:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_41 2024-04-23T21:49:09.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:09.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:09.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:09.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:09.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=5614], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:09.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=5614], [object_number=5, tag_tid=1, entry_tid=5613], [object_number=4, tag_tid=1, entry_tid=5612], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:09.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:09.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:09.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:09.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=5614' '!=' 'object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:09.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:09.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:49:09.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_41 2024-04-23T21:49:09.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:49:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_41 ']' 2024-04-23T21:49:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_41' 2024-04-23T21:49:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_41 2024-04-23T21:49:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:09.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:09.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:09.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:09.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:09.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_41 2024-04-23T21:49:09.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:09.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:09.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:09.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:09.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:09.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:09.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:09.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:09.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:09.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:09.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:09.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:09.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:09.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:09.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:09.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:09.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:09.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:09.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:09.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:09.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_41 2024-04-23T21:49:10.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_41'] 2024-04-23T21:49:10.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:10.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:10.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:10.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_41 2024-04-23T21:49:10.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:10.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:10.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:10.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:10.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:10.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:10.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:10.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:10.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:10.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:10.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:10.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:10.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:10.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:10.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:10.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:10.014 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:10.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:10.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:10.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:10.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:10.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:10.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:10.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:10.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:10.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:10.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:10.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:10.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:10.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:10.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:10.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:10.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:10.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_41 2024-04-23T21:49:10.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:10.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_41 2024-04-23T21:49:10.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:10.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:10.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:10.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_41 '' 2024-04-23T21:49:10.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:10.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:10.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:10.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:10.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_41.status 2024-04-23T21:49:10.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_41.status 2024-04-23T21:49:10.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:10.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:10.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_41 2024-04-23T21:49:10.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:10.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:10.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:10.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:10.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=5614], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:10.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=5614], [object_number=5, tag_tid=1, entry_tid=5613], [object_number=4, tag_tid=1, entry_tid=5612], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:10.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:10.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:10.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_41 2024-04-23T21:49:10.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:10.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:10.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:10.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_41 '..*' 2024-04-23T21:49:10.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:10.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:10.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:10.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:10.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_41.status 2024-04-23T21:49:10.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_41.status 2024-04-23T21:49:10.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:10.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:10.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_41 2024-04-23T21:49:10.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:10.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:10.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:10.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:10.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=5614], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:10.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=5614], [object_number=5, tag_tid=1, entry_tid=5613], [object_number=4, tag_tid=1, entry_tid=5612], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:10.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:10.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:10.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:10.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=5614' '!=' 'object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:10.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:10.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:49:11.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_41 2024-04-23T21:49:11.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:11.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:11.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:49:11.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:11.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_41 ']' 2024-04-23T21:49:11.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_41' 2024-04-23T21:49:11.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_41 2024-04-23T21:49:11.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:11.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:11.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:11.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:11.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:11.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:11.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:11.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_41 2024-04-23T21:49:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:11.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:11.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:11.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:11.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:11.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:11.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:11.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:11.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:11.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:11.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:11.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:11.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:11.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:11.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:11.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:11.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:11.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:11.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:11.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:11.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:11.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:11.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:11.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_41 2024-04-23T21:49:11.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_41'] 2024-04-23T21:49:11.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:11.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:11.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:11.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_41 2024-04-23T21:49:11.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:11.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:11.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:11.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:11.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:11.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:11.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:11.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:11.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:11.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:11.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:11.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:11.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:11.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:11.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:11.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:11.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:11.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_41 2024-04-23T21:49:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_41 2024-04-23T21:49:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:12.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:12.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_41 '' 2024-04-23T21:49:12.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:12.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:12.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:12.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:12.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_41.status 2024-04-23T21:49:12.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_41.status 2024-04-23T21:49:12.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:12.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:12.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_41 2024-04-23T21:49:12.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:12.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:12.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:12.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:12.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=5614], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:12.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=5614], [object_number=5, tag_tid=1, entry_tid=5613], [object_number=4, tag_tid=1, entry_tid=5612], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:12.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:12.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:12.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_41 2024-04-23T21:49:12.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:12.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:12.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:12.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_41 '..*' 2024-04-23T21:49:12.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:12.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:12.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:12.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:12.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_41.status 2024-04-23T21:49:12.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_41.status 2024-04-23T21:49:12.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:12.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:12.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_41 2024-04-23T21:49:12.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:12.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:12.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:12.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:12.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=5614], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:12.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=5614], [object_number=5, tag_tid=1, entry_tid=5613], [object_number=4, tag_tid=1, entry_tid=5612], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:12.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:12.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:12.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:12.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=5614' '!=' 'object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:12.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:12.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:49:14.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_41 2024-04-23T21:49:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:49:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_41 ']' 2024-04-23T21:49:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_41' 2024-04-23T21:49:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_41 2024-04-23T21:49:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:14.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:14.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:14.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:14.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:14.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:14.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_41 2024-04-23T21:49:14.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:14.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:14.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:14.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:14.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:14.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:14.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:14.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:14.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:14.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:14.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:14.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:14.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:14.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:14.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:14.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:14.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:14.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:14.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:14.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_41 2024-04-23T21:49:14.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_41'] 2024-04-23T21:49:14.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:14.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:14.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:14.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_41 2024-04-23T21:49:14.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:14.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:14.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:14.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:14.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:14.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:14.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:14.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:14.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:14.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:14.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:14.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:14.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:14.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:14.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:14.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:14.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:14.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:14.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:14.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:14.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:14.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:14.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:14.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:14.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:14.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:14.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:14.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:14.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:14.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:14.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:14.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:14.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:14.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_41 2024-04-23T21:49:14.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:14.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_41 2024-04-23T21:49:14.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:14.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:14.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:14.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_41 '' 2024-04-23T21:49:14.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:14.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:14.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:14.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:14.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_41.status 2024-04-23T21:49:14.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_41.status 2024-04-23T21:49:14.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:14.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:14.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_41 2024-04-23T21:49:14.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:14.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:14.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:14.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:14.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=5614], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:14.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=5614], [object_number=5, tag_tid=1, entry_tid=5613], [object_number=4, tag_tid=1, entry_tid=5612], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:14.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_41 2024-04-23T21:49:14.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:14.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:14.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:14.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_41 '..*' 2024-04-23T21:49:14.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:14.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:14.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:14.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:14.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_41.status 2024-04-23T21:49:14.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_41.status 2024-04-23T21:49:14.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:14.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:14.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_41 2024-04-23T21:49:14.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:14.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:14.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:14.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:14.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=5614], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:14.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=6, tag_tid=1, entry_tid=5614], [object_number=5, tag_tid=1, entry_tid=5613], [object_number=4, tag_tid=1, entry_tid=5612], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:14.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:14.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:14.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:14.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=6, tag_tid=1, entry_tid=5614' '!=' 'object_number=6, tag_tid=1, entry_tid=5614' 2024-04-23T21:49:14.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:14.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:49:16.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_41 2024-04-23T21:49:16.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:16.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:16.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:49:16.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:16.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_41 ']' 2024-04-23T21:49:16.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_41' 2024-04-23T21:49:16.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_41 2024-04-23T21:49:16.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:16.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:16.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:16.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:16.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:16.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:16.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:16.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:16.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:16.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_41 2024-04-23T21:49:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:16.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:16.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:16.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:16.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:16.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:16.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:16.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:16.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:16.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:16.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_41 2024-04-23T21:49:16.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_41'] 2024-04-23T21:49:16.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:16.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:16.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:16.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_41 2024-04-23T21:49:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:16.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:16.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:16.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:16.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:16.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:16.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_41 2024-04-23T21:49:16.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:16.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_41 2024-04-23T21:49:16.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:16.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:16.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:16.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_41 '' 2024-04-23T21:49:16.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:16.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:16.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:16.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:16.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_41.status 2024-04-23T21:49:16.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_41.status 2024-04-23T21:49:16.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:16.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:16.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_41 2024-04-23T21:49:16.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:16.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:16.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:16.736 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:16.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=5614], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:16.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=5614], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:16.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:16.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:16.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_41 2024-04-23T21:49:16.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:16.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:16.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:16.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_41 '..*' 2024-04-23T21:49:16.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:16.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:16.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:49:16.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:16.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_41.status 2024-04-23T21:49:16.742 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_41.status 2024-04-23T21:49:16.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:16.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:16.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_41 2024-04-23T21:49:16.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:16.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:16.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:16.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:16.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=5614], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:16.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=5614], [object_number=7, tag_tid=1, entry_tid=5611]]], state=connected] 2024-04-23T21:49:16.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.status 2024-04-23T21:49:16.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:16.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:16.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:16.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_41 snap 2024-04-23T21:49:16.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:16.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:16.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:49:16.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:16.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:49:16.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:49:16.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:49:17.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_41@snap 2024-04-23T21:49:17.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_41': 2024-04-23T21:49:17.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:49:17.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:49:17.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:49:17.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 10b5b3ce56ec 2024-04-23T21:49:17.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.10b5b3ce56ec 2024-04-23T21:49:17.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:49:17.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:49:17.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:49:17.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:49:17.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:38:36 2024 2024-04-23T21:49:17.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:38:36 2024 2024-04-23T21:49:17.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:39:37 2024 2024-04-23T21:49:17.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:49:17.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 10b5b3ce56ec 2024-04-23T21:49:17.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:49:17.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:49:17.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: ad73b364-e9fb-4b7b-a849-0295abac8367 2024-04-23T21:49:17.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:49:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_41 snap 2024-04-23T21:49:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:49:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:49:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.export 2024-04-23T21:49:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_41.export 2024-04-23T21:49:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_41.export 2024-04-23T21:49:17.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_41@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.export 2024-04-23T21:49:17.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:49:17.986 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_41@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_41.export 2024-04-23T21:49:18.394 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:49:18.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_41.export 2024-04-23T21:49:18.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_41.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_41.export 2024-04-23T21:49:18.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:18.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:49:18.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_42 2024-04-23T21:49:18.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_42 snap 2024-04-23T21:49:18.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:49:18.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:18.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:49:18.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:18.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_42@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:49:19.147 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:49:19.146+0000 7f5860606640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 2 2024-04-23T21:49:21.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_42 2024-04-23T21:49:21.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:21.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:21.137 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:49:21.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_42 started 2024-04-23T21:49:21.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:21.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:21.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:49:21.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:49:21.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:49:21.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:21.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:49:22.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_42 started 2024-04-23T21:49:22.138 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:22.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:22.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:49:22.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:49:22.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:49:22.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:49:22.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_42 2024-04-23T21:49:22.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:49:22.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:49:22.140 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:22.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:22.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:49:22.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:49:22.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:22.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:49:22.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:22.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:49:22.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:22.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_42 2024-04-23T21:49:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:49:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:49:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:49:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:22.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:49:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:49:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:49:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:49:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:49:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:49:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:49:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:49:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:49:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:49:22.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:22.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:22.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:22.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:22.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_42 2024-04-23T21:49:22.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_42'] 2024-04-23T21:49:22.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:49:22.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_42 2024-04-23T21:49:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:49:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:22.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:49:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:49:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:49:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:49:22.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:49:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:49:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:49:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:49:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:49:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:49:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:49:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:49:22.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:22.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:22.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:22.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:22.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_42 2024-04-23T21:49:22.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_42'] 2024-04-23T21:49:22.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:49:22.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:22.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:22.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_42 2024-04-23T21:49:22.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:49:22.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:22.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:22.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:22.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:22.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:49:22.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:22.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:22.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:22.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:22.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:49:22.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:22.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:22.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:22.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:22.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:49:22.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:49:22.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:49:22.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:22.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:49:22.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:49:22.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:49:22.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:22.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:22.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:22.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:49:22.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:49:22.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:49:22.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:49:22.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:22.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:22.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:22.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:22.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_42 2024-04-23T21:49:22.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_42'] 2024-04-23T21:49:22.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:49:22.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:22.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:22.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_42 2024-04-23T21:49:22.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:49:22.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:22.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:22.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:22.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:22.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:49:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:49:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:49:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:49:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:49:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:49:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:49:22.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:49:22.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:22.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:22.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:22.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:49:22.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:49:22.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:49:22.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:49:22.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:22.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:22.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:22.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:22.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_42 2024-04-23T21:49:22.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:49:22.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:49:22.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:49:22.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:49:22.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:49:22.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:49:22.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:49:22.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:22.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_42 2024-04-23T21:49:22.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:49:22.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:49:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:49:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:49:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_42 2024-04-23T21:49:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:49:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:49:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:49:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:49:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:49:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:49:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:49:22.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_42 2024-04-23T21:49:22.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:22.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:22.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:49:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_42 ']' 2024-04-23T21:49:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_42' 2024-04-23T21:49:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_42 2024-04-23T21:49:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:22.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:22.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_42 2024-04-23T21:49:22.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:22.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:22.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:22.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:22.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:22.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:22.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:22.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:22.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:22.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:22.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:22.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:22.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:22.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:22.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:22.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:22.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:22.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:22.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:22.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_42 2024-04-23T21:49:22.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_42'] 2024-04-23T21:49:22.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:22.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:22.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:22.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_42 2024-04-23T21:49:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:22.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:22.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:22.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:22.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:22.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:22.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:22.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:22.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:22.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:22.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:22.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:22.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:22.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:22.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:22.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:22.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:22.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:22.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:22.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:22.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:22.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:22.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_42 2024-04-23T21:49:22.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_42'] 2024-04-23T21:49:22.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:22.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:22.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:22.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_42 2024-04-23T21:49:22.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:49:22.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:22.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:22.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:22.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:49:22.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:22.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:22.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:22.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:49:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:49:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:49:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:22.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:22.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:22.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:22.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:22.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:22.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:22.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:23.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:23.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:23.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:23.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_42 2024-04-23T21:49:23.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_42'] 2024-04-23T21:49:23.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:23.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:23.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:23.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_42 2024-04-23T21:49:23.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:49:23.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:23.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:23.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:23.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:49:23.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:23.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:23.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:23.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:49:23.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:23.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:23.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:23.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:49:23.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:23.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:23.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:49:23.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:23.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:23.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:23.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:23.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:23.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:23.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:23.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:23.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:23.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:23.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:23.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:23.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:23.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:23.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_42 2024-04-23T21:49:23.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:23.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_42 2024-04-23T21:49:23.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:23.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:23.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_42 2024-04-23T21:49:23.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_42 '' 2024-04-23T21:49:23.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:23.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:23.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_42 2024-04-23T21:49:23.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:23.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_42.status 2024-04-23T21:49:23.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_42.status 2024-04-23T21:49:23.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:23.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.status 2024-04-23T21:49:23.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_42 2024-04-23T21:49:23.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.status 2024-04-23T21:49:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:49:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:49:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:23.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10439], [object_number=10, tag_tid=1, entry_tid=10438]]], state=connected] 2024-04-23T21:49:23.245 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=10439], [object_number=10, tag_tid=1, entry_tid=10438], [object_number=9, tag_tid=1, entry_tid=10437], [object_number=8, tag_tid=1, entry_tid=10436]]], state=connected] 2024-04-23T21:49:23.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.status 2024-04-23T21:49:23.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:49:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_42 2024-04-23T21:49:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_42 2024-04-23T21:49:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_42 '..*' 2024-04-23T21:49:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_42 2024-04-23T21:49:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_42.status 2024-04-23T21:49:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_42.status 2024-04-23T21:49:23.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:23.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.status 2024-04-23T21:49:23.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_42 2024-04-23T21:49:23.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.status 2024-04-23T21:49:23.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:49:23.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:49:23.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:23.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10439], [object_number=10, tag_tid=1, entry_tid=10438]]], state=connected] 2024-04-23T21:49:23.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=11, tag_tid=1, entry_tid=10439], [object_number=10, tag_tid=1, entry_tid=10438], [object_number=9, tag_tid=1, entry_tid=10437], [object_number=8, tag_tid=1, entry_tid=10436]]], state=connected] 2024-04-23T21:49:23.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.status 2024-04-23T21:49:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=11, tag_tid=1, entry_tid=10439' 2024-04-23T21:49:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=11, tag_tid=1, entry_tid=10439' 2024-04-23T21:49:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=10439' '!=' '' 2024-04-23T21:49:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:49:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=11, tag_tid=1, entry_tid=10439' = '' 2024-04-23T21:49:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=11, tag_tid=1, entry_tid=10439' 2024-04-23T21:49:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:49:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:49:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:23.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:49:23.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=11, tag_tid=1, entry_tid=10439' 2024-04-23T21:49:23.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:49:23.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:23.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:49:23.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:49:23.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:49:23.300 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:23.300 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6231843360187379) > weight (0.25) 2024-04-23T21:49:23.300 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.43888978763004116) > weight (0.25) 2024-04-23T21:49:23.300 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2024-04-23T21:49:23.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2024-04-23T21:49:23.301 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2024-04-23T21:49:23.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:49:23.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=11, tag_tid=1, entry_tid=10439' 2024-04-23T21:49:23.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:49:23.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:49:23.302+0000 7f3c51b8e640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 78359) UID: 1000 2024-04-23T21:49:23.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:23.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=10439 2024-04-23T21:49:23.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 10439 2024-04-23T21:49:23.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:49:23.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:23.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:49:23.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_42 2024-04-23T21:49:23.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:23.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:49:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_42 ']' 2024-04-23T21:49:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_42' 2024-04-23T21:49:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_42 2024-04-23T21:49:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:23.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:23.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:23.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:23.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_42 2024-04-23T21:49:23.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:23.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:23.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:23.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:23.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:23.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:23.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:23.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_42 2024-04-23T21:49:23.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_42'] 2024-04-23T21:49:23.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:23.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:23.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:23.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_42 2024-04-23T21:49:23.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:23.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:23.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:23.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:23.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:23.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:23.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:23.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:23.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:23.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:23.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:23.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:23.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:23.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:23.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:23.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:23.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:23.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:23.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:23.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:23.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:23.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:23.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:23.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:23.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:23.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:23.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:23.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:23.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:23.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:23.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:23.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_42 2024-04-23T21:49:23.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_42'] 2024-04-23T21:49:23.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:23.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:23.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:23.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_42 2024-04-23T21:49:23.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:49:23.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:23.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:23.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:23.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:49:23.712 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:23.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:23.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:23.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:49:23.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:23.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:23.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:23.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:49:23.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:23.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:23.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:49:23.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:23.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:23.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:23.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:23.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:23.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:23.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:23.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:23.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:23.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:23.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:23.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:23.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:23.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:23.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:23.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:23.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:23.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_42 2024-04-23T21:49:23.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2024-04-23T21:49:23.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:23.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_42 2024-04-23T21:49:23.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:49:23.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:23.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:23.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:23.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:49:23.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:23.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:23.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:23.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:49:23.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:23.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:23.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:23.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:49:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:49:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:23.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:23.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:23.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:23.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:23.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:23.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_42 2024-04-23T21:49:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_42 2024-04-23T21:49:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:23.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_42 2024-04-23T21:49:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_42 '' 2024-04-23T21:49:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_42 2024-04-23T21:49:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:23.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_42.status 2024-04-23T21:49:23.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:23.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_42.status 2024-04-23T21:49:23.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.status 2024-04-23T21:49:23.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_42 2024-04-23T21:49:23.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.status 2024-04-23T21:49:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:49:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:49:23.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:23.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10439], [object_number=10, tag_tid=1, entry_tid=10438]]], state=connected] 2024-04-23T21:49:23.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10439], [object_number=10, tag_tid=1, entry_tid=10438]]], state=connected] 2024-04-23T21:49:23.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.status 2024-04-23T21:49:23.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:49:23.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_42 2024-04-23T21:49:23.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:23.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:23.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_42 2024-04-23T21:49:23.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_42 '..*' 2024-04-23T21:49:23.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:23.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:23.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_42 2024-04-23T21:49:23.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:23.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_42.status 2024-04-23T21:49:23.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_42.status 2024-04-23T21:49:23.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:23.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.status 2024-04-23T21:49:23.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_42 2024-04-23T21:49:23.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.status 2024-04-23T21:49:24.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:49:24.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:49:24.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:24.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10439], [object_number=10, tag_tid=1, entry_tid=10438]]], state=connected] 2024-04-23T21:49:24.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10439], [object_number=10, tag_tid=1, entry_tid=10438]]], state=connected] 2024-04-23T21:49:24.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.status 2024-04-23T21:49:24.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:49:24.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:49:24.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:24.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_42 snap 2024-04-23T21:49:24.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:24.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:24.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:49:24.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:24.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:49:24.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:49:24.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:49:25.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_42@snap 2024-04-23T21:49:25.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_42': 2024-04-23T21:49:25.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:49:25.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:49:25.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:49:25.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 116ec7458442 2024-04-23T21:49:25.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.116ec7458442 2024-04-23T21:49:25.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:49:25.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:49:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:49:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:49:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:38:45 2024 2024-04-23T21:49:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:38:45 2024 2024-04-23T21:49:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:40:50 2024 2024-04-23T21:49:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:49:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 116ec7458442 2024-04-23T21:49:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:49:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:49:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 57037404-0324-47bc-8c82-d2732b3194a9 2024-04-23T21:49:25.058 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:49:25.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:25.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_42 snap 2024-04-23T21:49:25.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:25.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:49:25.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:25.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:49:25.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.export 2024-04-23T21:49:25.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_42.export 2024-04-23T21:49:25.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_42.export 2024-04-23T21:49:25.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_42@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.export 2024-04-23T21:49:25.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:49:25.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_42@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_42.export 2024-04-23T21:49:25.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:49:25.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_42.export 2024-04-23T21:49:25.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_42.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_42.export 2024-04-23T21:49:25.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:25.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:49:25.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_43 2024-04-23T21:49:25.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_43 snap 2024-04-23T21:49:25.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:49:25.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:25.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:49:25.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:25.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_43@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:49:26.301 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2024-04-23T21:49:26.301 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:49:26.302 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2024-04-23T21:49:26.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2024-04-23T21:49:26.302 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2024-04-23T21:49:26.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2024-04-23T21:49:26.305 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2024-04-23T21:49:27.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_43 2024-04-23T21:49:27.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:27.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:27.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:49:27.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_43 started 2024-04-23T21:49:27.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:27.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:27.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:49:27.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:49:27.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:49:27.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:27.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:49:28.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_43 started 2024-04-23T21:49:28.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:28.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:28.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:49:28.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:49:28.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:49:28.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:49:28.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_43 2024-04-23T21:49:28.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:49:28.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:49:28.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:28.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:28.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:49:28.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:49:28.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:28.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:49:28.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:28.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:49:28.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:28.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_43 2024-04-23T21:49:28.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:49:28.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:28.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:28.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:28.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:49:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:49:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:49:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:49:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:49:28.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:28.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:49:28.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:49:28.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:49:28.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:28.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:28.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:28.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:49:28.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:49:28.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:49:28.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:49:28.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:28.178 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:28.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:28.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:28.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_43 2024-04-23T21:49:28.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:28.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:49:28.266 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:28.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:28.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_43 2024-04-23T21:49:28.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:49:28.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:28.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:28.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:28.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:28.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:49:28.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:28.269 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:28.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:28.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:28.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:49:28.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:28.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:28.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:28.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:28.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:49:28.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:49:28.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:49:28.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:28.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:49:28.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:49:28.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:49:28.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:28.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:28.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:28.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:49:28.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:49:28.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:49:28.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:49:28.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:28.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:28.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:28.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:28.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_43 2024-04-23T21:49:28.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:28.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:49:28.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:28.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:28.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_43 2024-04-23T21:49:28.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:49:28.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:28.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:28.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:28.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:28.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:49:28.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:28.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:28.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:28.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:28.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:49:28.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:28.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:28.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:28.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:28.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:49:28.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:49:28.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:49:28.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:28.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:49:28.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:49:28.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:49:28.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:28.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:28.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:28.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:49:28.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:49:28.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:49:28.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:49:28.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:28.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:28.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:28.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:28.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_43 2024-04-23T21:49:28.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:28.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:49:28.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:28.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:28.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_43 2024-04-23T21:49:28.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:49:28.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:28.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:28.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:28.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:28.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:49:28.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:28.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:28.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:28.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:28.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:49:28.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:28.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:28.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:28.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:28.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:49:28.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:49:28.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:49:28.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:28.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:49:28.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:49:28.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:49:28.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:28.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:28.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:28.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:49:28.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:49:28.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:49:28.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:49:28.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:28.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:28.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:28.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:28.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_43 2024-04-23T21:49:28.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:49:28.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:49:28.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:49:28.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:49:28.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:49:28.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:49:28.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:49:28.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:28.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_43 2024-04-23T21:49:28.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:49:28.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:49:28.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:28.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:49:28.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:49:28.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_43 2024-04-23T21:49:28.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:49:28.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:49:28.586 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:28.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:49:28.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:49:28.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:49:28.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:49:28.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:28.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:49:28.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_43 2024-04-23T21:49:28.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:28.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:28.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:49:28.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:28.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_43 ']' 2024-04-23T21:49:28.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_43' 2024-04-23T21:49:28.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_43 2024-04-23T21:49:28.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:28.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:28.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:28.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:28.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:28.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:28.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:28.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:28.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:28.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:28.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_43 2024-04-23T21:49:28.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:28.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:28.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:28.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:28.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:28.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:28.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:28.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:28.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:28.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:28.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:28.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:28.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:28.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:28.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:28.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:28.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:28.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:28.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:28.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:28.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:28.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:28.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:28.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:28.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:28.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:28.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:28.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:28.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:28.793 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:28.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:28.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:28.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:28.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:28.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:28.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:28.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:28.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:28.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_43 2024-04-23T21:49:28.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:28.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:28.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:28.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:28.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:28.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:28.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:28.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:28.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:28.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:28.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:28.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:28.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:28.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:28.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:28.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:28.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:28.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:28.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:28.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:28.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:28.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:28.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:28.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:28.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:28.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_43 2024-04-23T21:49:28.992 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:49:28.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:28.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:28.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:28.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:49:28.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:28.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:28.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:28.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:49:28.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:28.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:28.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:28.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:49:28.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:28.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:49:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:28.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:29.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:29.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:29.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:29.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:29.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:29.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:29.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:29.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:29.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_43 2024-04-23T21:49:29.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:49:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:49:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:49:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:49:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:49:29.088 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:29.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:29.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:29.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:29.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:29.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:29.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:29.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:29.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:29.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:29.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:29.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:29.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:29.089 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:29.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:29.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:29.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:29.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:29.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:29.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_43 2024-04-23T21:49:29.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:29.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:29.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:29.195 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_43 '' 2024-04-23T21:49:29.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:29.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:29.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:29.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:29.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_43.status 2024-04-23T21:49:29.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_43.status 2024-04-23T21:49:29.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:29.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:29.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_43 2024-04-23T21:49:29.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:29.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:29.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:29.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:29.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:29.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6097], [object_number=4, tag_tid=1, entry_tid=6096], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:29.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:29.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:29.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_43 2024-04-23T21:49:29.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:29.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:29.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:29.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_43 '..*' 2024-04-23T21:49:29.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:29.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:29.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:29.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:29.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_43.status 2024-04-23T21:49:29.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_43.status 2024-04-23T21:49:29.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:29.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:29.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_43 2024-04-23T21:49:29.244 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:29.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:29.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:29.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:29.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:29.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6097], [object_number=4, tag_tid=1, entry_tid=6096], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:29.288 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:29.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:29.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:29.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=6097' '!=' '' 2024-04-23T21:49:29.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:49:29.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=6097' = '' 2024-04-23T21:49:29.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:29.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:29.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:49:29.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:29.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:49:29.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:29.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:49:29.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:29.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:49:29.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:29.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:49:29.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:29.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:49:29.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:29.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:49:29.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:29.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=6097 2024-04-23T21:49:29.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 6097 2024-04-23T21:49:29.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:49:29.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:29.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:49:29.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_43 2024-04-23T21:49:29.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:29.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:29.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:49:29.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:29.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_43 ']' 2024-04-23T21:49:29.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_43' 2024-04-23T21:49:29.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_43 2024-04-23T21:49:29.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:29.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:29.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:29.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:29.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:29.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:29.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:29.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:29.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:29.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:29.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_43 2024-04-23T21:49:29.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:29.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:29.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:29.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:29.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:29.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:29.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:29.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:29.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:29.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:29.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:29.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:29.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:29.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_43 2024-04-23T21:49:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:29.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:29.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:29.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:29.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:29.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:29.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:29.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:29.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:29.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:29.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:29.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:29.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:29.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:29.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:29.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:29.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:29.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:29.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:29.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:29.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:29.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:29.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:29.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:29.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:29.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:29.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_43 2024-04-23T21:49:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:49:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:29.704 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:49:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:49:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:49:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:49:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:29.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:29.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:29.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:29.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:29.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:29.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:29.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:29.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:29.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:29.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:29.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:29.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:29.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:29.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:29.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:29.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:29.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:29.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:29.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:29.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:29.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:29.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_43 2024-04-23T21:49:29.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:49:29.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:29.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:29.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:29.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:49:29.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:29.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:29.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:29.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:49:29.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:29.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:29.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:49:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:49:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:29.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:29.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:29.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:29.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:29.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:29.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:29.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:29.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:29.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:29.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_43 2024-04-23T21:49:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_43 '' 2024-04-23T21:49:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_43.status 2024-04-23T21:49:29.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_43.status 2024-04-23T21:49:29.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:29.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:29.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_43 2024-04-23T21:49:29.905 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:29.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:29.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:29.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:29.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:29.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6097], [object_number=4, tag_tid=1, entry_tid=6096], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:29.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:29.951 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_43 2024-04-23T21:49:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_43 '..*' 2024-04-23T21:49:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_43.status 2024-04-23T21:49:29.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_43.status 2024-04-23T21:49:29.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:29.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:29.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_43 2024-04-23T21:49:29.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:29.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:29.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:29.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:29.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:29.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6097], [object_number=4, tag_tid=1, entry_tid=6096], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:29.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:29.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:29.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:29.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=6097' '!=' 'object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:29.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:29.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:49:30.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_43 2024-04-23T21:49:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:49:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_43 ']' 2024-04-23T21:49:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_43' 2024-04-23T21:49:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_43 2024-04-23T21:49:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:30.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:30.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:30.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:30.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_43 2024-04-23T21:49:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:30.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:30.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:30.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:30.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:30.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:30.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:30.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:30.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:30.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:30.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:30.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:30.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:30.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:30.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:30.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:30.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:30.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:30.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:30.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:30.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:30.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:30.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:30.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:30.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:30.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:30.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_43 2024-04-23T21:49:30.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:30.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:30.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:30.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:30.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:30.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:30.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:30.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:30.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:30.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:30.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:30.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:30.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:30.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:30.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:30.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:30.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:30.523 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:30.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:30.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:30.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:30.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_43 2024-04-23T21:49:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:49:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:30.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:49:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:49:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:49:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:49:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:30.607 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:30.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:30.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:30.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:30.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:30.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:30.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:30.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:30.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:30.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:30.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_43 2024-04-23T21:49:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:49:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:49:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:49:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:49:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:30.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:49:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:30.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:30.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:30.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:30.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:30.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:30.806 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:30.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_43 2024-04-23T21:49:30.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_43 '' 2024-04-23T21:49:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_43.status 2024-04-23T21:49:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_43.status 2024-04-23T21:49:30.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:30.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:30.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_43 2024-04-23T21:49:30.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:30.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:30.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:30.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:30.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:30.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6097], [object_number=4, tag_tid=1, entry_tid=6096], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:30.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:30.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_43 2024-04-23T21:49:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_43 '..*' 2024-04-23T21:49:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_43.status 2024-04-23T21:49:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_43.status 2024-04-23T21:49:30.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:30.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:30.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_43 2024-04-23T21:49:30.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6097], [object_number=4, tag_tid=1, entry_tid=6096], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:30.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:30.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:30.903 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:30.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=6097' '!=' 'object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:30.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:30.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:49:31.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_43 2024-04-23T21:49:31.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:31.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:31.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:49:31.705 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:31.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_43 ']' 2024-04-23T21:49:31.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_43' 2024-04-23T21:49:31.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_43 2024-04-23T21:49:31.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:31.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:31.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:31.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:31.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:31.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:31.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:31.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:31.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:31.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:31.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_43 2024-04-23T21:49:31.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:31.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:31.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:31.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:31.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:31.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:31.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:31.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:31.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:31.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:31.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:31.727 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:31.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:31.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:31.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:31.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:31.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_43 2024-04-23T21:49:31.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:31.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:31.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:31.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:31.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:31.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:31.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:31.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:31.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:31.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:31.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:31.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:31.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:31.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:31.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:31.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:31.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:31.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:31.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:31.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:31.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:31.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:31.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:31.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:31.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:31.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:31.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:31.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:31.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:31.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:31.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:31.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:31.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:31.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:31.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:31.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:31.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:31.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:31.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_43 2024-04-23T21:49:31.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:49:31.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:31.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:49:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:49:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:49:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:49:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:31.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:31.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:31.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:31.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:31.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:31.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:31.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:31.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:31.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:31.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:31.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:31.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:31.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:31.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:31.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:31.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:31.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:32.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:32.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:32.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:32.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:32.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_43 2024-04-23T21:49:32.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:49:32.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:32.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:32.009 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:32.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:49:32.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:32.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:32.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:32.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:49:32.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:32.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:32.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:32.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:49:32.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:32.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:32.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:49:32.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:32.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:32.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:32.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:32.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:32.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:32.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:32.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:32.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:32.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_43 2024-04-23T21:49:32.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:32.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:32.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:32.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_43 '' 2024-04-23T21:49:32.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:32.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:32.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:32.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:32.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_43.status 2024-04-23T21:49:32.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_43.status 2024-04-23T21:49:32.116 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:32.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:32.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_43 2024-04-23T21:49:32.117 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:32.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:32.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:32.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:32.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:32.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6097], [object_number=4, tag_tid=1, entry_tid=6096], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:32.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:32.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:32.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_43 2024-04-23T21:49:32.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:32.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:32.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:32.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_43 '..*' 2024-04-23T21:49:32.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:32.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:32.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:32.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:32.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_43.status 2024-04-23T21:49:32.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_43.status 2024-04-23T21:49:32.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:32.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:32.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_43 2024-04-23T21:49:32.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:32.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:32.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:32.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:32.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:32.207 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6097], [object_number=4, tag_tid=1, entry_tid=6096], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:32.209 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:32.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:32.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:32.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=6097' '!=' 'object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:32.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:32.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:49:33.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_43 2024-04-23T21:49:33.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:33.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:33.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:49:33.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:33.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_43 ']' 2024-04-23T21:49:33.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_43' 2024-04-23T21:49:33.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_43 2024-04-23T21:49:33.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:33.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:33.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:33.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:33.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:33.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:33.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:33.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_43 2024-04-23T21:49:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:33.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:33.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:33.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:33.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:33.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:33.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:33.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:33.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:33.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:33.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:33.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:33.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:33.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:33.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:33.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:33.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:33.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:33.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:33.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:33.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:33.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:33.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_43 2024-04-23T21:49:33.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:33.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:33.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:33.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:33.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:33.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:33.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:33.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:33.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:33.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:33.920 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:33.921 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:33.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:33.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:33.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:33.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:33.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:33.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:34.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:34.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:34.017 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:34.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:34.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_43 2024-04-23T21:49:34.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:49:34.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:34.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:34.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:49:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:49:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:49:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:49:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:34.021 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:34.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:34.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:34.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:34.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:34.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:34.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:34.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:34.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:34.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:34.022 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:34.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:34.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:34.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:34.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:34.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:34.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:34.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:34.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:34.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_43 2024-04-23T21:49:34.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:49:34.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:34.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:34.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:34.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:49:34.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:34.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:34.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:34.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:49:34.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:34.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:34.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:34.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:49:34.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:34.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:34.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:49:34.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:34.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:34.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:34.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:34.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:34.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:34.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:34.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:34.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:34.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:34.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:34.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:34.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:34.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:34.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:34.141 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:34.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:34.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:34.228 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:34.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_43 2024-04-23T21:49:34.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:34.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:34.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:34.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_43 '' 2024-04-23T21:49:34.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:34.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:34.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:34.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:34.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_43.status 2024-04-23T21:49:34.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_43.status 2024-04-23T21:49:34.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:34.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:34.230 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_43 2024-04-23T21:49:34.231 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:34.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:34.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:34.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:34.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:34.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6097], [object_number=4, tag_tid=1, entry_tid=6096], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:34.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:34.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_43 2024-04-23T21:49:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_43 '..*' 2024-04-23T21:49:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_43.status 2024-04-23T21:49:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_43.status 2024-04-23T21:49:34.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:34.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:34.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_43 2024-04-23T21:49:34.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:34.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:34.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:34.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:34.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:34.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=1, entry_tid=6097], [object_number=4, tag_tid=1, entry_tid=6096], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:34.321 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=1, entry_tid=6097' '!=' 'object_number=5, tag_tid=1, entry_tid=6097' 2024-04-23T21:49:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:34.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:49:36.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_43 2024-04-23T21:49:36.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:49:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_43 ']' 2024-04-23T21:49:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_43' 2024-04-23T21:49:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_43 2024-04-23T21:49:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:36.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:36.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:36.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:36.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:36.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:36.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_43 2024-04-23T21:49:36.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:36.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:36.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:36.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:36.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:36.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:36.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:36.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:36.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:36.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:36.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:36.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:36.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:36.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:36.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:36.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:36.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:36.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:36.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:36.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:36.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:36.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:36.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:36.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:36.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:36.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:36.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:36.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:36.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:36.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:36.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:36.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_43 2024-04-23T21:49:36.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:36.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:36.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:36.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:36.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:36.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:36.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:36.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:36.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:36.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:36.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:36.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:36.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:36.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:36.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:36.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:36.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:36.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:36.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:36.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:36.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:36.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:36.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_43 2024-04-23T21:49:36.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:49:36.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:36.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:49:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:49:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:49:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:49:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:36.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:36.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:36.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:36.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:36.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:36.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:36.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:36.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:36.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:36.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:36.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:36.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:36.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:36.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:36.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:36.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:36.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2024-04-23T21:49:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:36.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_43 2024-04-23T21:49:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:49:36.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:49:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:49:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:49:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:49:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:36.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:36.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:36.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:36.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:36.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:36.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:36.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:36.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:36.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:36.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:36.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:36.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:36.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:36.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:36.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:36.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_43 2024-04-23T21:49:36.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:36.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_43 2024-04-23T21:49:36.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:36.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:36.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:36.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_43 '' 2024-04-23T21:49:36.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:36.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:36.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:36.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:36.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_43.status 2024-04-23T21:49:36.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_43.status 2024-04-23T21:49:36.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:36.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:36.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_43 2024-04-23T21:49:36.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:36.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:36.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:36.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:36.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:36.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:36.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:36.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:36.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_43 2024-04-23T21:49:36.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:36.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:36.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:36.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_43 '..*' 2024-04-23T21:49:36.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:36.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:36.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:49:36.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:36.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_43.status 2024-04-23T21:49:36.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_43.status 2024-04-23T21:49:36.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:36.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:36.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_43 2024-04-23T21:49:36.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:49:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:49:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:36.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:36.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=6095], [object_number=6, tag_tid=1, entry_tid=6094]]], state=connected] 2024-04-23T21:49:36.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.status 2024-04-23T21:49:36.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:36.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:49:36.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:36.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_43 snap 2024-04-23T21:49:36.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:36.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:36.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:49:36.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:36.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:49:36.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:49:36.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:49:37.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_43@snap 2024-04-23T21:49:37.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_43': 2024-04-23T21:49:37.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:49:37.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:49:37.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:49:37.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 11b8f461c2fb 2024-04-23T21:49:37.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.11b8f461c2fb 2024-04-23T21:49:37.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:49:37.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:49:37.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:49:37.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:49:37.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:38:56 2024 2024-04-23T21:49:37.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:38:56 2024 2024-04-23T21:49:37.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:38:56 2024 2024-04-23T21:49:37.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:49:37.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 11b8f461c2fb 2024-04-23T21:49:37.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:49:37.895 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:49:37.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: aeb90a21-0957-4205-8215-7e0553fb6f2c 2024-04-23T21:49:37.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:49:37.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:37.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_43 snap 2024-04-23T21:49:37.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:37.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:49:37.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:37.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:49:37.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.export 2024-04-23T21:49:37.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_43.export 2024-04-23T21:49:37.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_43.export 2024-04-23T21:49:37.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_43@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.export 2024-04-23T21:49:38.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:49:38.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_43@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_43.export 2024-04-23T21:49:38.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:49:38.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_43.export 2024-04-23T21:49:38.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_43.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_43.export 2024-04-23T21:49:38.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:38.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:49:38.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_44 2024-04-23T21:49:38.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_44 snap 2024-04-23T21:49:38.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:49:38.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:38.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:49:38.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:38.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_44@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:49:39.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_44 2024-04-23T21:49:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:49:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_44 started 2024-04-23T21:49:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:49:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:49:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:49:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:39.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:49:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_44 started 2024-04-23T21:49:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:49:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:49:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:49:40.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:49:40.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_44 2024-04-23T21:49:40.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:49:40.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:49:40.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:40.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:40.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:49:40.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:49:40.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:40.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:49:40.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:40.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:49:40.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:40.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_44 2024-04-23T21:49:40.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:49:40.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:40.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:40.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:40.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:49:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:49:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:49:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:49:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:49:40.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:40.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:49:40.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:49:40.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:49:40.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:40.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:40.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:40.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:49:40.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:49:40.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:49:40.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:49:40.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:40.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:40.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:40.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:40.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_44 2024-04-23T21:49:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:49:40.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:40.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:40.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_44 2024-04-23T21:49:40.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:49:40.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:40.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:40.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:40.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:40.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:49:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:49:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:49:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:49:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:49:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:49:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:49:40.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:49:40.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:40.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:40.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:40.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:49:40.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:49:40.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:49:40.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:49:40.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:40.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:40.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:40.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:40.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_44 2024-04-23T21:49:40.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:40.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:49:40.381 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:40.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:40.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_44 2024-04-23T21:49:40.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:49:40.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:40.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:40.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:40.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:40.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:49:40.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:40.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:40.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:40.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:40.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:49:40.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:40.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:40.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:40.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:40.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:49:40.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:49:40.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:49:40.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:40.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:49:40.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:49:40.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:49:40.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:40.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:40.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:40.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:49:40.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:49:40.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:49:40.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:49:40.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:40.401 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:40.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:40.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:40.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_44 2024-04-23T21:49:40.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:40.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:49:40.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_44 2024-04-23T21:49:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:3 2024-04-23T21:49:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:49:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:40.484 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:49:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:3 2024-04-23T21:49:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:49:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:49:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:3 2024-04-23T21:49:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:49:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:49:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:40.485 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:40.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:49:40.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:49:40.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=3 2024-04-23T21:49:40.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=3 2024-04-23T21:49:40.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:40.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:40.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:40.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:40.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_44 2024-04-23T21:49:40.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:49:40.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:49:40.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:49:40.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:49:40.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:49:40.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:49:40.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:49:40.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:40.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_44 2024-04-23T21:49:40.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:49:40.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:49:40.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:40.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:49:40.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:49:40.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_44 2024-04-23T21:49:40.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:49:40.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:49:40.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:40.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:49:40.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:49:40.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:49:40.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:49:40.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:40.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:49:40.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_44 2024-04-23T21:49:40.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:40.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:40.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:49:40.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:40.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_44 ']' 2024-04-23T21:49:40.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_44' 2024-04-23T21:49:40.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_44 2024-04-23T21:49:40.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:40.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:40.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:40.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:40.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:40.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:40.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:40.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:40.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_44 2024-04-23T21:49:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:40.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:40.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:40.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:40.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:40.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:40.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:40.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:40.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:40.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:40.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:40.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:40.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:40.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:40.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:40.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:40.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:40.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:40.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:40.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:40.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:40.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_44 2024-04-23T21:49:40.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:40.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:40.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:40.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:40.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:40.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:40.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:40.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:40.899 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:40.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:40.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:40.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:40.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:40.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:40.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:40.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:40.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:40.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_44 2024-04-23T21:49:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:49:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:49:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:49:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:49:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:40.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:40.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:49:40.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:40.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:40.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:40.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:40.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:40.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:40.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:40.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:40.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:40.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:40.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:40.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:40.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:40.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:41.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:41.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:41.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:41.015 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:41.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:41.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:41.099 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:41.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:41.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_44 2024-04-23T21:49:41.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:49:41.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:41.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:41.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:41.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:49:41.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:41.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:41.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:41.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:49:41.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:41.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:41.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:41.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:49:41.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:41.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:41.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:49:41.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:41.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:41.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:41.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:41.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:41.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:41.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:41.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:41.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:41.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:41.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:41.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:41.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:41.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:41.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:41.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:41.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_44 2024-04-23T21:49:41.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:41.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:41.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:41.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_44 '' 2024-04-23T21:49:41.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_44.status 2024-04-23T21:49:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_44.status 2024-04-23T21:49:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_44 2024-04-23T21:49:41.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:49:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:49:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:41.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:41.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:49:41.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_44 2024-04-23T21:49:41.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:41.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:41.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:41.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_44 '..*' 2024-04-23T21:49:41.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:41.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:41.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:41.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:41.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_44.status 2024-04-23T21:49:41.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_44.status 2024-04-23T21:49:41.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:41.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:41.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_44 2024-04-23T21:49:41.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:41.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:49:41.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:49:41.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:41.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:41.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:41.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:41.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:41.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=3, entry_tid=1' -a 'object_number=1, tag_tid=3, entry_tid=1' = 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:41.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=2, entry_tid=1' '!=' '' 2024-04-23T21:49:41.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:49:41.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=2, entry_tid=1' = '' 2024-04-23T21:49:41.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:41.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:49:41.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:49:41.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:41.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:49:41.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:41.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:49:41.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:41.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:49:41.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:49:41.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:49:41.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:41.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:49:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:49:41.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=1 2024-04-23T21:49:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 1 2024-04-23T21:49:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:49:41.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:41.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:49:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_44 2024-04-23T21:49:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:49:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_44 ']' 2024-04-23T21:49:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_44' 2024-04-23T21:49:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_44 2024-04-23T21:49:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:41.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:41.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:41.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:41.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:41.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:41.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:41.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_44 2024-04-23T21:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:41.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:41.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:41.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:41.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:41.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:41.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:41.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:41.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:41.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:41.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_44 2024-04-23T21:49:41.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:41.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:41.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:41.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:41.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:41.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:41.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:41.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:41.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:41.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:41.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:41.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:41.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:41.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:41.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:41.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:41.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:41.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:41.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_44 2024-04-23T21:49:41.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:49:41.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:41.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:41.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:41.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:49:41.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:41.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:41.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:41.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:49:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:49:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:49:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:41.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:41.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:41.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:41.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:41.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:41.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:41.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:41.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:41.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:41.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:41.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:41.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_44 2024-04-23T21:49:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:49:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:41.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:49:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:49:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:49:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:49:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:41.812 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:41.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:41.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:41.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:41.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:41.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:41.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:41.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:41.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:41.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:41.813 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:41.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:41.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:41.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:41.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:41.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:41.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_44 2024-04-23T21:49:41.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:41.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:41.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:41.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_44 '' 2024-04-23T21:49:41.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:41.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:41.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:41.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:41.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_44.status 2024-04-23T21:49:41.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_44.status 2024-04-23T21:49:41.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:41.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:41.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_44 2024-04-23T21:49:41.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:41.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:49:41.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:49:41.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:41.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:41.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:41.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:49:41.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_44 2024-04-23T21:49:41.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:41.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:41.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:41.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_44 '..*' 2024-04-23T21:49:41.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:41.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:41.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:41.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:41.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_44.status 2024-04-23T21:49:41.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_44.status 2024-04-23T21:49:41.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:41.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:41.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_44 2024-04-23T21:49:41.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:42.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:49:42.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:49:42.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:42.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:42.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:42.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:42.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:42.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=3, entry_tid=1' -a 'object_number=1, tag_tid=3, entry_tid=1' = 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:42.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=2, entry_tid=1' '!=' 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:42.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:42.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:49:42.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_44 2024-04-23T21:49:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:49:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_44 ']' 2024-04-23T21:49:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_44' 2024-04-23T21:49:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_44 2024-04-23T21:49:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:42.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:42.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:42.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:42.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:42.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_44 2024-04-23T21:49:42.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:42.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:42.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:42.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:42.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:42.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:42.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:42.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:42.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:42.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:42.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:42.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:42.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:42.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:42.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:42.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:42.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:42.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:42.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:42.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:42.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:42.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:42.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:42.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:42.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:42.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:42.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:42.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:42.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:42.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:42.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:42.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:42.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:42.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:42.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:42.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:42.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:42.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:42.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_44 2024-04-23T21:49:42.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:42.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:42.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:42.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:42.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:42.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:42.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:42.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:42.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:42.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:42.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:42.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:42.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:42.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:42.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:42.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:42.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:42.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:42.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:42.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:42.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:42.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:42.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:42.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:42.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:42.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:42.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:42.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:42.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:42.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:42.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:42.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:42.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:42.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:42.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:42.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:42.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:42.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:42.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_44 2024-04-23T21:49:42.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:49:42.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:42.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:42.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:42.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:49:42.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:42.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:42.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:42.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:49:42.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:42.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:42.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:42.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:49:42.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:42.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:42.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:49:42.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:42.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:42.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:42.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:42.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:42.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:42.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:42.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:42.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:42.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:42.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:42.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:42.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:42.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:42.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:42.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:42.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:42.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:42.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:42.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:42.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:42.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_44 2024-04-23T21:49:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:49:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:49:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:49:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:42.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:49:42.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:42.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:42.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:49:42.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:42.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:42.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:42.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:42.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:42.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:42.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:42.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:42.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:42.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:42.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:42.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:42.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:42.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:42.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:42.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:42.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:42.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:42.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:42.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_44 2024-04-23T21:49:42.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:42.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:42.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:42.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_44 '' 2024-04-23T21:49:42.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:42.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:42.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:42.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:42.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_44.status 2024-04-23T21:49:42.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_44.status 2024-04-23T21:49:42.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:42.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:42.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_44 2024-04-23T21:49:42.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:42.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:49:42.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:49:42.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:42.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:42.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:42.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:42.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:49:42.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_44 2024-04-23T21:49:42.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:42.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:42.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:42.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_44 '..*' 2024-04-23T21:49:42.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:42.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:42.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:42.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:42.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_44.status 2024-04-23T21:49:42.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_44.status 2024-04-23T21:49:42.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:42.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:42.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_44 2024-04-23T21:49:42.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:42.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:49:42.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:49:42.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:42.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:42.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:42.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:42.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:42.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=3, entry_tid=1' -a 'object_number=1, tag_tid=3, entry_tid=1' = 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:42.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=2, entry_tid=1' '!=' 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:42.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:42.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:49:43.715 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_44 2024-04-23T21:49:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:49:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_44 ']' 2024-04-23T21:49:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_44' 2024-04-23T21:49:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_44 2024-04-23T21:49:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:43.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:43.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:43.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:43.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:43.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_44 2024-04-23T21:49:43.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:43.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:43.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:43.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:43.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:43.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:43.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:43.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:43.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:43.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:43.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:43.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:43.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:43.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:43.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:43.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:43.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:43.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:43.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:43.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:43.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:43.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:43.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:43.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:43.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:43.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:43.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:43.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:43.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:43.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:43.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:43.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:43.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:43.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:43.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_44 2024-04-23T21:49:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:43.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:43.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:43.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:43.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:43.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:43.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:43.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:43.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:43.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:43.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:43.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:43.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:43.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:43.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:43.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:43.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_44 2024-04-23T21:49:43.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:49:43.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:43.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:43.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:49:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:49:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:49:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:49:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:43.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:43.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:43.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:43.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:43.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:43.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:44.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:44.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:44.020 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:44.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:44.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_44 2024-04-23T21:49:44.023 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:49:44.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:44.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:44.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:44.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:49:44.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:44.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:44.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:44.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:49:44.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:44.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:44.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:44.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:49:44.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:44.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:49:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:44.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:44.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:44.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:44.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:44.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_44 2024-04-23T21:49:44.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:44.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:44.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:44.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_44 '' 2024-04-23T21:49:44.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:44.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:44.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:44.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:44.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_44.status 2024-04-23T21:49:44.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_44.status 2024-04-23T21:49:44.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:44.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:44.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_44 2024-04-23T21:49:44.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:44.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:49:44.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:49:44.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:44.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:44.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:44.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:44.177 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:49:44.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_44 2024-04-23T21:49:44.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:44.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:44.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:44.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_44 '..*' 2024-04-23T21:49:44.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:44.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:44.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:44.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:44.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_44.status 2024-04-23T21:49:44.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_44.status 2024-04-23T21:49:44.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:44.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:44.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_44 2024-04-23T21:49:44.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:44.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:49:44.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:49:44.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:44.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:44.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:44.223 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:44.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:44.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=3, entry_tid=1' -a 'object_number=1, tag_tid=3, entry_tid=1' = 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:44.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=2, entry_tid=1' '!=' 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:44.226 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:49:45.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_44 2024-04-23T21:49:45.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:45.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:45.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:49:45.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:45.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_44 ']' 2024-04-23T21:49:45.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_44' 2024-04-23T21:49:45.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_44 2024-04-23T21:49:45.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:45.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:45.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:45.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:45.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:45.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:45.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:45.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:45.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:45.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:45.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_44 2024-04-23T21:49:45.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:45.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:45.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:45.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:45.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:45.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:45.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:45.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:45.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:45.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:45.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:45.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:45.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:45.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:45.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:45.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:45.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:45.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:45.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:45.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:45.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:45.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:45.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:45.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:45.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:45.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:45.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:45.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:45.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:45.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:45.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:45.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:45.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:45.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:45.937 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:45.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:45.938 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:45.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:45.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_44 2024-04-23T21:49:45.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:45.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:45.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:45.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:45.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:45.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:45.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:45.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:45.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:45.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:45.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:45.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:45.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:45.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:45.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:45.943 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:45.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:45.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:45.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:45.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:45.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:46.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:46.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:46.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:46.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_44 2024-04-23T21:49:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:49:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:49:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:49:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:49:46.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:49:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:46.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:46.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:46.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:46.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:46.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:46.144 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_44 2024-04-23T21:49:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:49:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:49:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:49:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:49:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:49:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:46.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:46.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:46.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:46.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:46.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:46.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:46.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:46.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_44 2024-04-23T21:49:46.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:46.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:46.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:46.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_44 '' 2024-04-23T21:49:46.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:46.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:46.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:46.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:46.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_44.status 2024-04-23T21:49:46.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_44.status 2024-04-23T21:49:46.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:46.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:46.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_44 2024-04-23T21:49:46.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:46.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:49:46.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:49:46.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:46.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:46.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:46.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:46.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:49:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_44 2024-04-23T21:49:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_44 '..*' 2024-04-23T21:49:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_44.status 2024-04-23T21:49:46.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_44.status 2024-04-23T21:49:46.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:46.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:46.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_44 2024-04-23T21:49:46.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:46.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:49:46.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:49:46.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:46.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:46.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=2, entry_tid=1], [object_number=0, tag_tid=2, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:46.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:46.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:46.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=3, entry_tid=1' -a 'object_number=1, tag_tid=3, entry_tid=1' = 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:46.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=1, tag_tid=2, entry_tid=1' '!=' 'object_number=1, tag_tid=2, entry_tid=1' 2024-04-23T21:49:46.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:46.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:49:48.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_44 2024-04-23T21:49:48.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:48.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:48.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:49:48.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:48.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_44 ']' 2024-04-23T21:49:48.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_44' 2024-04-23T21:49:48.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_44 2024-04-23T21:49:48.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:48.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:48.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:48.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:48.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:48.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:48.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:48.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:48.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:48.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:48.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_44 2024-04-23T21:49:48.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:48.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:48.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:48.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:48.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:48.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:48.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:48.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:48.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:48.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:48.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:48.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:48.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:48.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:48.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:48.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:48.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:48.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:48.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:48.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:48.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:48.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:48.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:48.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:48.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:48.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:48.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:48.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:48.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:48.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:48.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:48.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:48.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:48.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:48.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:48.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:48.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_44 2024-04-23T21:49:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:49:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:49:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:49:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:49:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:48.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:49:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:49:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:49:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:49:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:49:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:49:48.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:49:48.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:48.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:48.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:49:48.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:49:48.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:49:48.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:48.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:48.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:49:48.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:48.570 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:48.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:48.571 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:48.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:48.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_44 2024-04-23T21:49:48.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:49:48.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:48.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:48.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:49:48.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:49:48.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:48.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:49:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:49:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:49:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:49:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:49:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:49:48.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:48.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:48.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:48.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:49:48.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:49:48.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:48.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:48.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:49:48.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:49:48.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:49:48.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:48.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:48.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:49:48.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:48.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2024-04-23T21:49:48.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:49:48.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:49:48.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:48.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:3 rbd mirror flush mirror/image_44 2024-04-23T21:49:48.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:49:48.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:48.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:3 cluster instance 2024-04-23T21:49:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:3 2024-04-23T21:49:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=3 2024-04-23T21:49:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=3 2024-04-23T21:49:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:3 cluster1 2024-04-23T21:49:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:3 2024-04-23T21:49:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:48.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:3 local_cluster instance 2024-04-23T21:49:48.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:3 2024-04-23T21:49:48.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:48.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:48.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:48.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=3 2024-04-23T21:49:48.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 3 = cluster1:3 ']' 2024-04-23T21:49:48.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:48.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:48.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=3 2024-04-23T21:49:48.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=3 2024-04-23T21:49:48.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2024-04-23T21:49:48.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:48.691 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:48.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2024-04-23T21:49:48.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror flush mirror/image_44 2024-04-23T21:49:48.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:48.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_44 2024-04-23T21:49:48.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:48.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:48.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:48.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_44 '' 2024-04-23T21:49:48.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:48.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:48.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:48.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:48.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_44.status 2024-04-23T21:49:48.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_44.status 2024-04-23T21:49:48.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:48.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:48.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_44 2024-04-23T21:49:48.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:48.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:49:48.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:49:48.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:48.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:48.826 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:48.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:48.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:49:48.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_44 2024-04-23T21:49:48.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:48.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:48.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:48.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_44 '..*' 2024-04-23T21:49:48.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:48.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:48.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:49:48.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:48.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_44.status 2024-04-23T21:49:48.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_44.status 2024-04-23T21:49:48.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:48.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:48.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_44 2024-04-23T21:49:48.833 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:48.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 0 2024-04-23T21:49:48.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 0 2024-04-23T21:49:48.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:48.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:48.874 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=1, tag_tid=3, entry_tid=1], [object_number=0, tag_tid=3, entry_tid=0], [object_number=3, tag_tid=1, entry_tid=3379], [object_number=2, tag_tid=1, entry_tid=3378]]], state=connected] 2024-04-23T21:49:48.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.status 2024-04-23T21:49:48.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:49:48.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=1, tag_tid=3, entry_tid=1' -a 'object_number=1, tag_tid=3, entry_tid=1' = 'object_number=1, tag_tid=3, entry_tid=1' 2024-04-23T21:49:48.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:48.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_44 snap 2024-04-23T21:49:48.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:48.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:48.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:49:48.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:48.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:49:48.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:49:48.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:49:49.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_44@snap 2024-04-23T21:49:49.907 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.567098787290868) > weight (0.25) 2024-04-23T21:49:49.907 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.27564542649378465) > weight (0.25) 2024-04-23T21:49:49.907 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6949016689141342) > weight (0.25) 2024-04-23T21:49:49.908 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6041054319627913) > weight (0.25) 2024-04-23T21:49:49.908 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2024-04-23T21:49:49.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_44': 2024-04-23T21:49:49.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:49:49.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:49:49.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:49:49.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 12327670db6f 2024-04-23T21:49:49.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.12327670db6f 2024-04-23T21:49:49.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:49:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:49:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:49:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:49:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:39:03 2024 2024-04-23T21:49:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:39:03 2024 2024-04-23T21:49:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:40:04 2024 2024-04-23T21:49:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:49:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 12327670db6f 2024-04-23T21:49:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:49:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:49:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: f466019e-f921-4402-925e-4699481a917c 2024-04-23T21:49:49.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:49:49.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:49.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_44 snap 2024-04-23T21:49:49.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:49.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:49:49.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:49.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:49:49.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.export 2024-04-23T21:49:49.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_44.export 2024-04-23T21:49:49.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_44.export 2024-04-23T21:49:49.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_44@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.export 2024-04-23T21:49:50.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:49:50.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_44@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_44.export 2024-04-23T21:49:50.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:49:50.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_44.export 2024-04-23T21:49:50.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_44.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_44.export 2024-04-23T21:49:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:49:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_45 2024-04-23T21:49:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_45 snap 2024-04-23T21:49:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:49:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:49:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:50.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_45@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:49:52.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_45 2024-04-23T21:49:52.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:52.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:49:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_45 started 2024-04-23T21:49:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:49:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:49:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:49:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:49:53.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_45 started 2024-04-23T21:49:53.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:53.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:53.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:49:53.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:49:53.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:49:53.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:49:53.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_45 2024-04-23T21:49:53.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:49:53.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:49:53.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:53.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:53.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:49:53.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:49:53.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:53.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:49:53.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:53.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:49:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_45 2024-04-23T21:49:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:49:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:49:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:49:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:49:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:49:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:49:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:49:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:53.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:53.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:53.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:53.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_45 2024-04-23T21:49:53.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:49:53.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:49:53.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:49:53.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:49:53.311 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:49:53.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:49:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:49:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_45 2024-04-23T21:49:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:49:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:49:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:49:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:49:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_45 2024-04-23T21:49:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:49:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:49:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:53.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:49:53.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:49:53.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:49:53.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:49:53.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:53.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:49:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_45 2024-04-23T21:49:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:49:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_45 ']' 2024-04-23T21:49:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_45' 2024-04-23T21:49:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_45 2024-04-23T21:49:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:53.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:53.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:53.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_45 2024-04-23T21:49:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:53.518 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:53.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:53.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:53.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:53.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:53.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:53.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:53.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:53.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:53.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:53.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_45 2024-04-23T21:49:53.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:53.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_45 2024-04-23T21:49:53.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:53.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:53.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:49:53.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_45 '' 2024-04-23T21:49:53.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:53.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:53.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:49:53.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:53.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_45.status 2024-04-23T21:49:53.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_45.status 2024-04-23T21:49:53.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:53.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:53.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_45 2024-04-23T21:49:53.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:53.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:49:53.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:49:53.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:53.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13031], [object_number=14, tag_tid=1, entry_tid=13030]]], state=connected] 2024-04-23T21:49:53.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=13031], [object_number=14, tag_tid=1, entry_tid=13030], [object_number=13, tag_tid=1, entry_tid=13029], [object_number=12, tag_tid=1, entry_tid=13028]]], state=connected] 2024-04-23T21:49:53.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:53.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:49:53.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_45 2024-04-23T21:49:53.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:53.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:53.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:49:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_45 '..*' 2024-04-23T21:49:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:49:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_45.status 2024-04-23T21:49:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_45.status 2024-04-23T21:49:53.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_45 2024-04-23T21:49:53.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:53.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:49:53.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:49:53.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:53.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13031], [object_number=14, tag_tid=1, entry_tid=13030]]], state=connected] 2024-04-23T21:49:53.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=13031], [object_number=14, tag_tid=1, entry_tid=13030], [object_number=13, tag_tid=1, entry_tid=13029], [object_number=12, tag_tid=1, entry_tid=13028]]], state=connected] 2024-04-23T21:49:53.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:53.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=15, tag_tid=1, entry_tid=13031' 2024-04-23T21:49:53.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=15, tag_tid=1, entry_tid=13031' 2024-04-23T21:49:53.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=15, tag_tid=1, entry_tid=13031' '!=' '' 2024-04-23T21:49:53.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:49:53.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=15, tag_tid=1, entry_tid=13031' = '' 2024-04-23T21:49:53.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=15, tag_tid=1, entry_tid=13031' 2024-04-23T21:49:53.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:49:53.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:49:53.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:53.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:49:53.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=15, tag_tid=1, entry_tid=13031' 2024-04-23T21:49:53.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:49:53.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:53.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:49:53.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:49:53.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:49:53.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:53.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:49:53.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=15, tag_tid=1, entry_tid=13031' 2024-04-23T21:49:53.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:49:53.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:49:53.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=13031 2024-04-23T21:49:53.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 13031 2024-04-23T21:49:53.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:49:53.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:53.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:49:53.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_45 2024-04-23T21:49:53.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:53.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:53.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:49:53.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:53.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_45 ']' 2024-04-23T21:49:53.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_45' 2024-04-23T21:49:53.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_45 2024-04-23T21:49:53.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:53.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:53.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:53.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:53.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:53.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:53.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:53.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:53.979 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:53.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:53.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_45 2024-04-23T21:49:53.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:53.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:53.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:53.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:54.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:54.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:54.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:54.013 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_45 2024-04-23T21:49:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_45 2024-04-23T21:49:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:49:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_45 '' 2024-04-23T21:49:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:49:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:54.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_45.status 2024-04-23T21:49:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_45.status 2024-04-23T21:49:54.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_45 2024-04-23T21:49:54.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:54.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:49:54.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:49:54.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:54.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13031], [object_number=14, tag_tid=1, entry_tid=13030]]], state=connected] 2024-04-23T21:49:54.206 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=13031], [object_number=14, tag_tid=1, entry_tid=13030], [object_number=13, tag_tid=1, entry_tid=13029], [object_number=12, tag_tid=1, entry_tid=13028]]], state=connected] 2024-04-23T21:49:54.208 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:54.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:49:54.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_45 2024-04-23T21:49:54.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:54.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:54.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:49:54.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_45 '..*' 2024-04-23T21:49:54.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:54.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:54.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:49:54.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:54.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_45.status 2024-04-23T21:49:54.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_45.status 2024-04-23T21:49:54.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:54.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:54.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_45 2024-04-23T21:49:54.212 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:54.254 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:49:54.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:49:54.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:54.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13031], [object_number=14, tag_tid=1, entry_tid=13030]]], state=connected] 2024-04-23T21:49:54.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=13031], [object_number=14, tag_tid=1, entry_tid=13030], [object_number=13, tag_tid=1, entry_tid=13029], [object_number=12, tag_tid=1, entry_tid=13028]]], state=connected] 2024-04-23T21:49:54.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:54.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=15, tag_tid=1, entry_tid=13031' 2024-04-23T21:49:54.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=15, tag_tid=1, entry_tid=13031' 2024-04-23T21:49:54.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=15, tag_tid=1, entry_tid=13031' '!=' 'object_number=15, tag_tid=1, entry_tid=13031' 2024-04-23T21:49:54.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:54.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:49:54.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_45 2024-04-23T21:49:54.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:54.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:54.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:49:54.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:54.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_45 ']' 2024-04-23T21:49:54.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_45' 2024-04-23T21:49:54.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_45 2024-04-23T21:49:54.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:54.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:54.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:54.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:54.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:54.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:54.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:54.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:54.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:54.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:54.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_45 2024-04-23T21:49:54.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:54.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:54.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:54.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:54.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:54.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:54.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:54.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:54.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:54.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:54.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:54.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:54.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:54.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:54.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:54.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:54.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:54.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:54.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:54.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:54.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:54.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:54.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:54.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:54.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:54.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:54.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:54.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:54.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:54.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:54.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:54.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:54.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:54.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_45 2024-04-23T21:49:54.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:54.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_45 2024-04-23T21:49:54.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:54.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:54.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:49:54.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_45 '' 2024-04-23T21:49:54.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:54.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:54.843 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:49:54.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:54.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_45.status 2024-04-23T21:49:54.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_45.status 2024-04-23T21:49:54.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:54.844 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:54.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_45 2024-04-23T21:49:54.845 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:55.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:49:55.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:49:55.313 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:55.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13031], [object_number=14, tag_tid=1, entry_tid=13030]]], state=connected] 2024-04-23T21:49:55.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=13031], [object_number=14, tag_tid=1, entry_tid=13030], [object_number=13, tag_tid=1, entry_tid=13029], [object_number=12, tag_tid=1, entry_tid=13028]]], state=connected] 2024-04-23T21:49:55.314 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:55.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:49:55.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_45 2024-04-23T21:49:55.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:55.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:55.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:49:55.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_45 '..*' 2024-04-23T21:49:55.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:55.317 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:55.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:49:55.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:55.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_45.status 2024-04-23T21:49:55.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_45.status 2024-04-23T21:49:55.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:55.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:55.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_45 2024-04-23T21:49:55.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:55.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:49:55.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:49:55.361 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:55.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13031], [object_number=14, tag_tid=1, entry_tid=13030]]], state=connected] 2024-04-23T21:49:55.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=13031], [object_number=14, tag_tid=1, entry_tid=13030], [object_number=13, tag_tid=1, entry_tid=13029], [object_number=12, tag_tid=1, entry_tid=13028]]], state=connected] 2024-04-23T21:49:55.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:55.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=15, tag_tid=1, entry_tid=13031' 2024-04-23T21:49:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=15, tag_tid=1, entry_tid=13031' 2024-04-23T21:49:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=15, tag_tid=1, entry_tid=13031' '!=' 'object_number=15, tag_tid=1, entry_tid=13031' 2024-04-23T21:49:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:49:55.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:49:56.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_45 2024-04-23T21:49:56.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:56.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:56.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:49:56.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:49:56.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_45 ']' 2024-04-23T21:49:56.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_45' 2024-04-23T21:49:56.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_45 2024-04-23T21:49:56.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:49:56.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:56.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:56.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:49:56.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:49:56.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:56.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:49:56.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:49:56.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:49:56.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:49:56.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_45 2024-04-23T21:49:56.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:49:56.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:49:56.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:49:56.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:49:56.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:49:56.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:49:56.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:49:56.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:49:56.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:49:56.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:56.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:49:56.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:49:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:49:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:49:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:49:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:49:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:49:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:49:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:49:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:49:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:49:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:49:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:49:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:49:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:49:56.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:49:56.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:49:56.174 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:49:56.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:49:56.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_45 2024-04-23T21:49:56.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:56.284 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_45 2024-04-23T21:49:56.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:56.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:56.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:49:56.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_45 '' 2024-04-23T21:49:56.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:56.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:56.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:49:56.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:49:56.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_45.status 2024-04-23T21:49:56.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_45.status 2024-04-23T21:49:56.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:56.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:56.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_45 2024-04-23T21:49:56.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:49:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:49:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13031], [object_number=14, tag_tid=1, entry_tid=13030]]], state=connected] 2024-04-23T21:49:56.329 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13031], [object_number=14, tag_tid=1, entry_tid=13030]]], state=connected] 2024-04-23T21:49:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:49:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_45 2024-04-23T21:49:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:49:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_45 '..*' 2024-04-23T21:49:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:49:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:49:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:49:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:49:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_45.status 2024-04-23T21:49:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_45.status 2024-04-23T21:49:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:49:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_45 2024-04-23T21:49:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:56.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:49:56.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:49:56.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:49:56.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13031], [object_number=14, tag_tid=1, entry_tid=13030]]], state=connected] 2024-04-23T21:49:56.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13031], [object_number=14, tag_tid=1, entry_tid=13030]]], state=connected] 2024-04-23T21:49:56.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.status 2024-04-23T21:49:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:49:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:49:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_45 snap 2024-04-23T21:49:56.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:56.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:56.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:49:56.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:56.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:49:56.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:49:56.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:49:57.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_45@snap 2024-04-23T21:49:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_45': 2024-04-23T21:49:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:49:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:49:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:49:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 12322e711c2b 2024-04-23T21:49:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.12322e711c2b 2024-04-23T21:49:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:49:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:49:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:49:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:49:57.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:39:07 2024 2024-04-23T21:49:57.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:39:07 2024 2024-04-23T21:49:57.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:40:08 2024 2024-04-23T21:49:57.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:49:57.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 12322e711c2b 2024-04-23T21:49:57.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:49:57.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:49:57.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 2f8aad9c-04e5-48bf-aadc-966e8c29740c 2024-04-23T21:49:57.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:49:57.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:57.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_45 snap 2024-04-23T21:49:57.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:57.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:49:57.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:57.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:49:57.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.export 2024-04-23T21:49:57.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_45.export 2024-04-23T21:49:57.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_45.export 2024-04-23T21:49:57.636 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_45@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.export 2024-04-23T21:49:57.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:49:57.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_45@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_45.export 2024-04-23T21:49:58.018 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:49:58.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_45.export 2024-04-23T21:49:58.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_45.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_45.export 2024-04-23T21:49:58.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:49:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:49:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_46 2024-04-23T21:49:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_46 snap 2024-04-23T21:49:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:49:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:49:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:49:58.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_46@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:49:59.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_46 2024-04-23T21:49:59.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:59.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:59.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:49:59.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_46 started 2024-04-23T21:49:59.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:49:59.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:49:59.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:49:59.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:49:59.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:49:59.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:49:59.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:50:00.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_46 started 2024-04-23T21:50:00.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:00.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:00.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:50:00.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:50:00.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:50:00.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:50:00.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_46 2024-04-23T21:50:00.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:50:00.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:50:00.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:50:00.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:00.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:50:00.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:50:00.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:00.386 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:50:00.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:00.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:50:00.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:00.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_46 2024-04-23T21:50:00.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:50:00.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:50:00.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:00.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:00.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:00.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:50:00.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:00.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:00.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:00.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:00.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:50:00.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:50:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:50:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:50:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:50:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:50:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:50:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:00.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:00.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:50:00.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:50:00.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:50:00.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:50:00.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:00.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:00.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:00.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:00.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_46 2024-04-23T21:50:00.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_46'] 2024-04-23T21:50:00.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:50:00.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:00.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:00.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_46 2024-04-23T21:50:00.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:50:00.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:50:00.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:00.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:00.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:00.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:50:00.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:00.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:00.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:00.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:00.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:50:00.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:00.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:00.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:00.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:00.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:50:00.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:50:00.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:50:00.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:00.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:50:00.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:50:00.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:50:00.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:00.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:00.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:00.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:50:00.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:50:00.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:50:00.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:50:00.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_46 2024-04-23T21:50:00.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:50:00.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:50:00.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:50:00.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:50:00.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:50:00.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:50:00.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:50:00.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:00.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_46 2024-04-23T21:50:00.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:50:00.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:00.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:00.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:50:00.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:50:00.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_46 2024-04-23T21:50:00.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:50:00.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:00.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:00.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:50:00.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:50:00.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:50:00.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:50:00.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:00.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:50:00.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_46 2024-04-23T21:50:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:50:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_46 ']' 2024-04-23T21:50:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_46' 2024-04-23T21:50:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_46 2024-04-23T21:50:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:00.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:00.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:00.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:00.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:00.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_46 2024-04-23T21:50:00.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:00.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:00.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:00.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:00.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:00.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:00.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:00.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:00.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:00.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:00.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:00.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:00.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:00.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:00.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:00.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:00.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:00.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:00.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:00.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:00.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:00.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:00.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:00.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:00.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:00.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:00.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:00.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:00.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:00.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:00.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:00.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:00.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:00.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_46 2024-04-23T21:50:00.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_46'] 2024-04-23T21:50:00.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:00.919 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:00.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:00.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_46 2024-04-23T21:50:00.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:00.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:00.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:00.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:00.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:00.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:00.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:00.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:00.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:00.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:00.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:00.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:00.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:00.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:00.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:00.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:00.923 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:00.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:00.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:00.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:00.941 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_46 2024-04-23T21:50:01.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:01.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_46 2024-04-23T21:50:01.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:01.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:01.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:01.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_46 '' 2024-04-23T21:50:01.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:01.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:01.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:01.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:01.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_46.status 2024-04-23T21:50:01.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_46.status 2024-04-23T21:50:01.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:01.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:01.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_46 2024-04-23T21:50:01.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:01.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:01.072 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:01.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:01.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446]]], state=connected] 2024-04-23T21:50:01.073 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446], [object_number=5, tag_tid=1, entry_tid=7445], [object_number=4, tag_tid=1, entry_tid=7444]]], state=connected] 2024-04-23T21:50:01.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:01.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:01.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_46 2024-04-23T21:50:01.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:01.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:01.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:01.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_46 '..*' 2024-04-23T21:50:01.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:01.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:01.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_46.status 2024-04-23T21:50:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_46.status 2024-04-23T21:50:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_46 2024-04-23T21:50:01.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:01.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:01.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:01.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:01.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446]]], state=connected] 2024-04-23T21:50:01.120 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446], [object_number=5, tag_tid=1, entry_tid=7445], [object_number=4, tag_tid=1, entry_tid=7444]]], state=connected] 2024-04-23T21:50:01.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:01.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=7, tag_tid=1, entry_tid=7447' 2024-04-23T21:50:01.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=7, tag_tid=1, entry_tid=7447' 2024-04-23T21:50:01.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=7, tag_tid=1, entry_tid=7447' '!=' '' 2024-04-23T21:50:01.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:50:01.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=7, tag_tid=1, entry_tid=7447' = '' 2024-04-23T21:50:01.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=7, tag_tid=1, entry_tid=7447' 2024-04-23T21:50:01.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:01.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:50:01.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:50:01.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:50:01.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=7, tag_tid=1, entry_tid=7447' 2024-04-23T21:50:01.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:50:01.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:50:01.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:50:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:50:01.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:50:01.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:50:01.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=7, tag_tid=1, entry_tid=7447' 2024-04-23T21:50:01.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:50:01.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:50:01.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=7447 2024-04-23T21:50:01.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 7447 2024-04-23T21:50:01.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:50:01.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:01.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:50:01.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_46 2024-04-23T21:50:01.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:01.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:01.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:50:01.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:01.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_46 ']' 2024-04-23T21:50:01.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_46' 2024-04-23T21:50:01.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_46 2024-04-23T21:50:01.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:01.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:01.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:01.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:01.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:01.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:01.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:01.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:01.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:01.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:01.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_46 2024-04-23T21:50:01.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:01.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:01.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:01.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:01.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:01.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:01.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:01.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:01.354 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_46 2024-04-23T21:50:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_46'] 2024-04-23T21:50:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:01.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:01.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:01.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_46 2024-04-23T21:50:01.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:01.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:01.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:01.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:01.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:01.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:01.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:01.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:01.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:01.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:01.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_46 2024-04-23T21:50:01.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:01.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_46 2024-04-23T21:50:01.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:01.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:01.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:01.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_46 '' 2024-04-23T21:50:01.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:01.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:01.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:01.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:01.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_46.status 2024-04-23T21:50:01.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_46.status 2024-04-23T21:50:01.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:01.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:01.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_46 2024-04-23T21:50:01.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:01.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:01.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:01.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:01.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446]]], state=connected] 2024-04-23T21:50:01.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446], [object_number=5, tag_tid=1, entry_tid=7445], [object_number=4, tag_tid=1, entry_tid=7444]]], state=connected] 2024-04-23T21:50:01.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:01.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:01.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_46 2024-04-23T21:50:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_46 '..*' 2024-04-23T21:50:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_46.status 2024-04-23T21:50:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_46.status 2024-04-23T21:50:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:01.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:01.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_46 2024-04-23T21:50:01.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:01.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:01.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:01.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:01.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446]]], state=connected] 2024-04-23T21:50:01.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446], [object_number=5, tag_tid=1, entry_tid=7445], [object_number=4, tag_tid=1, entry_tid=7444]]], state=connected] 2024-04-23T21:50:01.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:01.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=7, tag_tid=1, entry_tid=7447' 2024-04-23T21:50:01.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=7, tag_tid=1, entry_tid=7447' 2024-04-23T21:50:01.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=7, tag_tid=1, entry_tid=7447' '!=' 'object_number=7, tag_tid=1, entry_tid=7447' 2024-04-23T21:50:01.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:01.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:50:02.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_46 2024-04-23T21:50:02.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:02.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:02.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:50:02.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:02.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_46 ']' 2024-04-23T21:50:02.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_46' 2024-04-23T21:50:02.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_46 2024-04-23T21:50:02.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:02.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:02.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:02.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:02.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:02.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:02.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:02.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:02.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:02.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:02.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_46 2024-04-23T21:50:02.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:02.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:02.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:02.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:02.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:02.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:02.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:02.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:02.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:02.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:02.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:02.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:02.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:02.045 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:02.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:02.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:02.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:02.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_46 2024-04-23T21:50:02.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_46'] 2024-04-23T21:50:02.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:02.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:02.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:02.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_46 2024-04-23T21:50:02.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:02.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:02.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:02.146 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:02.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:02.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:02.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:02.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:02.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:02.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:02.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:02.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:02.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:02.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:02.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:02.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:02.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:02.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:02.147 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:02.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:02.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:02.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:02.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:02.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_46 2024-04-23T21:50:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_46 2024-04-23T21:50:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_46 '' 2024-04-23T21:50:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_46.status 2024-04-23T21:50:02.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:02.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_46.status 2024-04-23T21:50:02.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:02.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_46 2024-04-23T21:50:02.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:02.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:02.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:02.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:02.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446]]], state=connected] 2024-04-23T21:50:02.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446], [object_number=5, tag_tid=1, entry_tid=7445], [object_number=4, tag_tid=1, entry_tid=7444]]], state=connected] 2024-04-23T21:50:02.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:02.301 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:02.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_46 2024-04-23T21:50:02.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:02.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:02.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:02.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_46 '..*' 2024-04-23T21:50:02.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:02.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_46.status 2024-04-23T21:50:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_46.status 2024-04-23T21:50:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_46 2024-04-23T21:50:02.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:02.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:02.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:02.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:02.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446]]], state=connected] 2024-04-23T21:50:02.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446], [object_number=5, tag_tid=1, entry_tid=7445], [object_number=4, tag_tid=1, entry_tid=7444]]], state=connected] 2024-04-23T21:50:02.347 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:02.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=7, tag_tid=1, entry_tid=7447' 2024-04-23T21:50:02.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=7, tag_tid=1, entry_tid=7447' 2024-04-23T21:50:02.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=7, tag_tid=1, entry_tid=7447' '!=' 'object_number=7, tag_tid=1, entry_tid=7447' 2024-04-23T21:50:02.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:02.349 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:50:03.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_46 2024-04-23T21:50:03.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:03.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:03.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:50:03.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:03.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_46 ']' 2024-04-23T21:50:03.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_46' 2024-04-23T21:50:03.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_46 2024-04-23T21:50:03.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:03.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:03.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:03.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:03.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:03.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:03.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:03.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:03.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:03.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:03.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_46 2024-04-23T21:50:03.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:03.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:03.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:03.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:03.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:03.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:03.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_46 2024-04-23T21:50:03.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_46'] 2024-04-23T21:50:03.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:03.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_46 2024-04-23T21:50:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:03.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:03.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:03.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:03.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:03.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:03.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:03.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:03.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:03.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:03.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:03.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:03.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:03.258 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:03.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:03.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:03.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:03.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:03.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:03.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:03.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:03.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:03.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:03.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:03.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_46 2024-04-23T21:50:03.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:03.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_46 2024-04-23T21:50:03.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:03.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:03.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:03.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_46 '' 2024-04-23T21:50:03.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:03.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:03.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:03.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:03.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_46.status 2024-04-23T21:50:03.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_46.status 2024-04-23T21:50:03.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:03.364 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:03.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_46 2024-04-23T21:50:03.365 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:03.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:03.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:03.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:03.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446]]], state=connected] 2024-04-23T21:50:03.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446], [object_number=5, tag_tid=1, entry_tid=7445], [object_number=4, tag_tid=1, entry_tid=7444]]], state=connected] 2024-04-23T21:50:03.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:03.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:03.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_46 2024-04-23T21:50:03.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:03.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:03.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:03.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_46 '..*' 2024-04-23T21:50:03.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:03.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:03.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:03.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:03.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_46.status 2024-04-23T21:50:03.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_46.status 2024-04-23T21:50:03.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:03.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:03.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_46 2024-04-23T21:50:03.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:03.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:03.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:03.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:03.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446]]], state=connected] 2024-04-23T21:50:03.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446], [object_number=5, tag_tid=1, entry_tid=7445], [object_number=4, tag_tid=1, entry_tid=7444]]], state=connected] 2024-04-23T21:50:03.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:03.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=7, tag_tid=1, entry_tid=7447' 2024-04-23T21:50:03.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=7, tag_tid=1, entry_tid=7447' 2024-04-23T21:50:03.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=7, tag_tid=1, entry_tid=7447' '!=' 'object_number=7, tag_tid=1, entry_tid=7447' 2024-04-23T21:50:03.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:03.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:50:05.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_46 2024-04-23T21:50:05.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:05.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:05.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:50:05.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:05.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_46 ']' 2024-04-23T21:50:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_46' 2024-04-23T21:50:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_46 2024-04-23T21:50:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:05.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:05.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:05.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:05.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_46 2024-04-23T21:50:05.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:05.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:05.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:05.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:05.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:05.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:05.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:05.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:05.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:05.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:05.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:05.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_46 2024-04-23T21:50:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_46'] 2024-04-23T21:50:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:05.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_46 2024-04-23T21:50:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:05.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:05.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:05.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:05.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:05.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:05.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:05.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:05.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:05.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:05.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:05.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:05.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:05.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:05.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:05.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:05.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:05.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:05.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:05.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:05.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:05.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:05.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:05.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:05.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:05.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:05.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_46 2024-04-23T21:50:05.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:05.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_46 2024-04-23T21:50:05.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:05.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:05.277 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:05.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_46 '' 2024-04-23T21:50:05.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:05.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:05.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:05.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:05.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_46.status 2024-04-23T21:50:05.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_46.status 2024-04-23T21:50:05.278 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:05.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:05.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_46 2024-04-23T21:50:05.279 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:05.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:05.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:05.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:05.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446]]], state=connected] 2024-04-23T21:50:05.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446]]], state=connected] 2024-04-23T21:50:05.322 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:05.323 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:05.324 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_46 2024-04-23T21:50:05.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:05.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:05.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:05.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_46 '..*' 2024-04-23T21:50:05.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:05.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:05.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:50:05.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:05.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_46.status 2024-04-23T21:50:05.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_46.status 2024-04-23T21:50:05.325 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:05.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:05.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_46 2024-04-23T21:50:05.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446]]], state=connected] 2024-04-23T21:50:05.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=7, tag_tid=1, entry_tid=7447], [object_number=6, tag_tid=1, entry_tid=7446]]], state=connected] 2024-04-23T21:50:05.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.status 2024-04-23T21:50:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=2, entry_tid=1' -a 'object_number=5, tag_tid=2, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_46 snap 2024-04-23T21:50:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:50:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:50:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:50:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:50:05.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:50:06.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_46@snap 2024-04-23T21:50:06.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_46': 2024-04-23T21:50:06.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:50:06.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:50:06.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:50:06.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 11b8fcaaebe8 2024-04-23T21:50:06.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.11b8fcaaebe8 2024-04-23T21:50:06.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:50:06.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:50:06.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:50:06.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:50:06.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:39:21 2024 2024-04-23T21:50:06.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:39:21 2024 2024-04-23T21:50:06.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:40:22 2024 2024-04-23T21:50:06.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:50:06.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 11b8fcaaebe8 2024-04-23T21:50:06.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:50:06.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:50:06.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: b6c89bf2-e4d2-4966-92bb-c3ab2186a049 2024-04-23T21:50:06.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:50:06.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_46 snap 2024-04-23T21:50:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:50:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:50:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:50:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.export 2024-04-23T21:50:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_46.export 2024-04-23T21:50:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_46.export 2024-04-23T21:50:06.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_46@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.export 2024-04-23T21:50:06.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:50:06.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_46@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_46.export 2024-04-23T21:50:06.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:50:06.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_46.export 2024-04-23T21:50:06.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_46.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_46.export 2024-04-23T21:50:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_47 2024-04-23T21:50:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_47 snap 2024-04-23T21:50:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:50:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:50:06.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_47@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:50:10.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_47 2024-04-23T21:50:10.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:10.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:10.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:50:10.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_47 started 2024-04-23T21:50:10.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:10.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:10.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:50:10.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:50:10.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:50:10.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:10.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:50:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_47 started 2024-04-23T21:50:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:50:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:50:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:50:11.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:50:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_47 2024-04-23T21:50:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:50:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:50:11.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:50:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:50:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:50:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:50:11.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:11.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:50:11.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:11.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_47 2024-04-23T21:50:11.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:50:11.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:50:11.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:11.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:11.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:11.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:50:11.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:11.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:11.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:11.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:50:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:50:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:50:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:50:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:50:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:50:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:50:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:11.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:11.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:50:11.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:50:11.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:50:11.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:50:11.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:11.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:11.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:11.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:11.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_47 2024-04-23T21:50:11.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:50:11.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:50:11.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:50:11.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:50:11.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:50:11.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:50:11.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:50:11.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:11.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_47 2024-04-23T21:50:11.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:50:11.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:11.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:11.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:50:11.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:50:11.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_47 2024-04-23T21:50:11.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:50:11.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:11.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:11.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:50:11.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:50:11.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:50:11.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:50:11.533 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:11.534 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:50:11.733 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_47 2024-04-23T21:50:11.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:11.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:11.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:50:11.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:11.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_47 ']' 2024-04-23T21:50:11.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_47' 2024-04-23T21:50:11.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_47 2024-04-23T21:50:11.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:11.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:11.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:11.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:11.734 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:11.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:11.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:11.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:11.737 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_47 2024-04-23T21:50:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:11.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:11.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:11.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:11.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:11.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:11.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:11.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:11.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:11.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:11.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:11.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_47 2024-04-23T21:50:11.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:11.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_47 2024-04-23T21:50:11.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:11.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:11.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:11.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_47 '' 2024-04-23T21:50:11.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:11.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:11.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:11.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:11.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_47.status 2024-04-23T21:50:11.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_47.status 2024-04-23T21:50:11.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:11.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:11.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_47 2024-04-23T21:50:11.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:11.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:11.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:11.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:11.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814]]], state=connected] 2024-04-23T21:50:11.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814], [object_number=13, tag_tid=1, entry_tid=13813], [object_number=12, tag_tid=1, entry_tid=13812]]], state=connected] 2024-04-23T21:50:11.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:11.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:11.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_47 2024-04-23T21:50:11.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:11.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:11.896 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:11.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_47 '..*' 2024-04-23T21:50:11.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:11.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:11.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:11.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:11.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_47.status 2024-04-23T21:50:11.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_47.status 2024-04-23T21:50:11.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:11.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:11.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_47 2024-04-23T21:50:11.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814]]], state=connected] 2024-04-23T21:50:11.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814], [object_number=13, tag_tid=1, entry_tid=13813], [object_number=12, tag_tid=1, entry_tid=13812]]], state=connected] 2024-04-23T21:50:11.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=15, tag_tid=1, entry_tid=13815' 2024-04-23T21:50:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=15, tag_tid=1, entry_tid=13815' 2024-04-23T21:50:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=15, tag_tid=1, entry_tid=13815' '!=' '' 2024-04-23T21:50:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:50:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=15, tag_tid=1, entry_tid=13815' = '' 2024-04-23T21:50:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=15, tag_tid=1, entry_tid=13815' 2024-04-23T21:50:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:50:11.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:50:11.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:50:11.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=15, tag_tid=1, entry_tid=13815' 2024-04-23T21:50:11.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:50:11.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:50:11.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:50:11.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:11.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:50:11.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:50:11.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:50:11.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=15, tag_tid=1, entry_tid=13815' 2024-04-23T21:50:11.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:50:11.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:50:11.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=13815 2024-04-23T21:50:11.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 13815 2024-04-23T21:50:11.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:50:11.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:11.950 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:50:12.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_47 2024-04-23T21:50:12.151 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:12.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:12.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:50:12.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:12.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_47 ']' 2024-04-23T21:50:12.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_47' 2024-04-23T21:50:12.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_47 2024-04-23T21:50:12.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:12.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:12.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:12.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:12.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:12.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:12.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:12.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:12.153 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_47 2024-04-23T21:50:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:12.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:12.157 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:12.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:12.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:12.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:12.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:12.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:12.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:12.158 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:12.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:12.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:12.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:12.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_47 2024-04-23T21:50:12.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:12.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_47 2024-04-23T21:50:12.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:12.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:12.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:12.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_47 '' 2024-04-23T21:50:12.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:12.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:12.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:12.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:12.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_47.status 2024-04-23T21:50:12.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_47.status 2024-04-23T21:50:12.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:12.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:12.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_47 2024-04-23T21:50:12.262 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:12.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:12.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:12.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:12.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814]]], state=connected] 2024-04-23T21:50:12.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814], [object_number=13, tag_tid=1, entry_tid=13813], [object_number=12, tag_tid=1, entry_tid=13812]]], state=connected] 2024-04-23T21:50:12.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:12.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:12.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_47 2024-04-23T21:50:12.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:12.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:12.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:12.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_47 '..*' 2024-04-23T21:50:12.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:12.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:12.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:12.309 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:12.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_47.status 2024-04-23T21:50:12.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_47.status 2024-04-23T21:50:12.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:12.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:12.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_47 2024-04-23T21:50:12.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:12.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:12.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:12.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:12.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814]]], state=connected] 2024-04-23T21:50:12.352 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814], [object_number=13, tag_tid=1, entry_tid=13813], [object_number=12, tag_tid=1, entry_tid=13812]]], state=connected] 2024-04-23T21:50:12.355 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=15, tag_tid=1, entry_tid=13815' 2024-04-23T21:50:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=15, tag_tid=1, entry_tid=13815' 2024-04-23T21:50:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=15, tag_tid=1, entry_tid=13815' '!=' 'object_number=15, tag_tid=1, entry_tid=13815' 2024-04-23T21:50:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:50:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_47 2024-04-23T21:50:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:50:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_47 ']' 2024-04-23T21:50:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_47' 2024-04-23T21:50:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_47 2024-04-23T21:50:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:12.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:12.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:12.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:12.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:12.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:12.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:12.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_47 2024-04-23T21:50:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:12.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:12.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:12.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:12.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:12.779 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:12.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:12.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:12.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_47 2024-04-23T21:50:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_47 2024-04-23T21:50:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_47 '' 2024-04-23T21:50:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:12.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:12.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:12.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_47.status 2024-04-23T21:50:12.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_47.status 2024-04-23T21:50:12.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:12.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:12.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_47 2024-04-23T21:50:12.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:12.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:12.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:12.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:12.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814]]], state=connected] 2024-04-23T21:50:12.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814], [object_number=13, tag_tid=1, entry_tid=13813], [object_number=12, tag_tid=1, entry_tid=13812]]], state=connected] 2024-04-23T21:50:12.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:12.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:12.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_47 2024-04-23T21:50:12.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:12.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:12.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:12.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_47 '..*' 2024-04-23T21:50:12.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_47.status 2024-04-23T21:50:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_47.status 2024-04-23T21:50:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:12.916 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_47 2024-04-23T21:50:12.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:12.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:12.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:12.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:12.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814]]], state=connected] 2024-04-23T21:50:12.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814], [object_number=13, tag_tid=1, entry_tid=13813], [object_number=12, tag_tid=1, entry_tid=13812]]], state=connected] 2024-04-23T21:50:12.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:12.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=15, tag_tid=1, entry_tid=13815' 2024-04-23T21:50:12.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=15, tag_tid=1, entry_tid=13815' 2024-04-23T21:50:12.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=15, tag_tid=1, entry_tid=13815' '!=' 'object_number=15, tag_tid=1, entry_tid=13815' 2024-04-23T21:50:12.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:12.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:50:13.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_47 2024-04-23T21:50:13.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:13.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:13.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:50:13.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:13.765 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_47 ']' 2024-04-23T21:50:13.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_47' 2024-04-23T21:50:13.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_47 2024-04-23T21:50:13.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:13.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:13.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:13.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:13.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:13.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:13.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:13.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:13.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:13.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:13.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_47 2024-04-23T21:50:13.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:13.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:13.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:13.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:13.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:13.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:13.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:13.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:13.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:13.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:13.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_47 2024-04-23T21:50:13.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:13.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_47 2024-04-23T21:50:13.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:13.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:13.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:13.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_47 '' 2024-04-23T21:50:13.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:13.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:13.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:13.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:13.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_47.status 2024-04-23T21:50:13.878 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_47.status 2024-04-23T21:50:13.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:13.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:13.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_47 2024-04-23T21:50:13.879 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:13.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:13.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:13.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:13.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814]]], state=connected] 2024-04-23T21:50:13.922 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814], [object_number=13, tag_tid=1, entry_tid=13813], [object_number=12, tag_tid=1, entry_tid=13812]]], state=connected] 2024-04-23T21:50:13.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_47 2024-04-23T21:50:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_47 '..*' 2024-04-23T21:50:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:13.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_47.status 2024-04-23T21:50:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_47.status 2024-04-23T21:50:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_47 2024-04-23T21:50:13.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:13.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:13.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:13.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:13.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814]]], state=connected] 2024-04-23T21:50:13.970 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814], [object_number=13, tag_tid=1, entry_tid=13813], [object_number=12, tag_tid=1, entry_tid=13812]]], state=connected] 2024-04-23T21:50:13.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:13.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=15, tag_tid=1, entry_tid=13815' 2024-04-23T21:50:13.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=15, tag_tid=1, entry_tid=13815' 2024-04-23T21:50:13.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=15, tag_tid=1, entry_tid=13815' '!=' 'object_number=15, tag_tid=1, entry_tid=13815' 2024-04-23T21:50:13.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:13.975 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:50:15.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_47 2024-04-23T21:50:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:50:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_47 ']' 2024-04-23T21:50:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_47' 2024-04-23T21:50:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_47 2024-04-23T21:50:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:15.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:15.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:15.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:15.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:15.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:15.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_47 2024-04-23T21:50:15.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:15.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:15.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:15.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:15.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:15.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:15.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:15.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:15.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:15.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:15.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:15.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:15.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:15.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:15.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:15.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:15.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:15.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:15.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:15.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_47 2024-04-23T21:50:15.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:15.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_47 2024-04-23T21:50:15.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:15.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:15.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:15.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_47 '' 2024-04-23T21:50:15.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:15.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:15.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:15.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:15.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_47.status 2024-04-23T21:50:15.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_47.status 2024-04-23T21:50:15.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:15.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:15.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_47 2024-04-23T21:50:15.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:15.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:15.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:15.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:15.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814]]], state=connected] 2024-04-23T21:50:15.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814]]], state=connected] 2024-04-23T21:50:15.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_47 2024-04-23T21:50:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_47 '..*' 2024-04-23T21:50:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:50:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_47.status 2024-04-23T21:50:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_47.status 2024-04-23T21:50:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_47 2024-04-23T21:50:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:15.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:15.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:15.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:15.794 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814]]], state=connected] 2024-04-23T21:50:15.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=13815], [object_number=14, tag_tid=1, entry_tid=13814]]], state=connected] 2024-04-23T21:50:15.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.status 2024-04-23T21:50:15.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:15.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:15.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:15.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_47 snap 2024-04-23T21:50:15.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:15.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:15.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:50:15.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:50:15.799 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:50:15.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:50:15.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:50:16.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_47@snap 2024-04-23T21:50:16.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_47': 2024-04-23T21:50:16.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:50:16.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:50:16.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:50:16.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 121d39dc6712 2024-04-23T21:50:16.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.121d39dc6712 2024-04-23T21:50:16.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:50:16.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:50:16.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:50:16.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:50:16.846 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:39:30 2024 2024-04-23T21:50:16.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:39:30 2024 2024-04-23T21:50:16.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:40:31 2024 2024-04-23T21:50:16.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:50:16.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 121d39dc6712 2024-04-23T21:50:16.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:50:16.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:50:16.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 8db98186-ef57-4051-8169-efd750a2785d 2024-04-23T21:50:16.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:50:16.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:16.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_47 snap 2024-04-23T21:50:16.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:16.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:50:16.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:50:16.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:50:16.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.export 2024-04-23T21:50:16.850 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_47.export 2024-04-23T21:50:16.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_47.export 2024-04-23T21:50:16.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_47@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.export 2024-04-23T21:50:17.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:50:17.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_47@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_47.export 2024-04-23T21:50:17.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:50:17.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_47.export 2024-04-23T21:50:17.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_47.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_47.export 2024-04-23T21:50:17.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:17.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:17.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_48 2024-04-23T21:50:17.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_48 snap 2024-04-23T21:50:17.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:17.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:17.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:50:17.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:50:17.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_48@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:50:19.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_48 2024-04-23T21:50:19.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:19.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:19.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:50:19.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_48 started 2024-04-23T21:50:19.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:19.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:19.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:50:19.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:50:19.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:50:19.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:19.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:50:20.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_48 started 2024-04-23T21:50:20.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:20.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:20.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:50:20.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:50:20.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:50:20.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:50:20.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_48 2024-04-23T21:50:20.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:50:20.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:50:20.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:50:20.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:20.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:50:20.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:50:20.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:20.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:50:20.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:50:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_48 2024-04-23T21:50:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:50:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:50:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:50:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:20.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:50:20.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:20.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:20.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:20.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:20.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:50:20.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:50:20.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:50:20.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:20.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:50:20.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:50:20.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:50:20.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:20.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:20.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:20.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:50:20.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:50:20.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:50:20.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:50:20.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:20.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:20.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:20.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:20.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_48 2024-04-23T21:50:20.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_48'] 2024-04-23T21:50:20.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:50:20.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:20.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_48 2024-04-23T21:50:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:50:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:50:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:50:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:50:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:20.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:50:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:50:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:50:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:50:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:50:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:50:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:50:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:50:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:50:20.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:50:20.541 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:20.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:20.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:20.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:20.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_48 2024-04-23T21:50:20.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:50:20.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:50:20.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:50:20.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:50:20.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:50:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:50:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:50:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_48 2024-04-23T21:50:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:50:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:50:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:50:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_48 2024-04-23T21:50:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:50:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:50:20.653 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:50:20.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:50:20.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:50:20.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:20.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:50:20.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_48 2024-04-23T21:50:20.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:20.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:20.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:50:20.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:20.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_48 ']' 2024-04-23T21:50:20.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_48' 2024-04-23T21:50:20.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_48 2024-04-23T21:50:20.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:20.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:20.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:20.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:20.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:20.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:20.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:20.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:20.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:20.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:20.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_48 2024-04-23T21:50:20.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:20.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:20.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:20.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:20.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:20.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:20.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:20.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:20.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:20.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:20.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:20.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:20.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:20.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:20.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:20.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:20.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:20.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:20.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:20.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:20.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:20.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:20.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:20.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:20.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:20.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:20.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:20.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:20.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:20.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:20.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:20.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:20.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:20.875 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_48 2024-04-23T21:50:20.952 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_48'] 2024-04-23T21:50:20.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:20.953 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:20.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:20.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_48 2024-04-23T21:50:20.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:20.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:20.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:20.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:20.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:20.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:20.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:20.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:20.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:20.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:20.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:20.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:20.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:20.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:20.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:20.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:20.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:20.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:20.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:20.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:20.974 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_48 2024-04-23T21:50:21.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:21.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_48 2024-04-23T21:50:21.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:21.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:21.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:21.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_48 '' 2024-04-23T21:50:21.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:21.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:21.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:21.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:21.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_48.status 2024-04-23T21:50:21.063 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_48.status 2024-04-23T21:50:21.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:21.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:21.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_48 2024-04-23T21:50:21.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:21.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:21.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:21.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:21.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706]]], state=connected] 2024-04-23T21:50:21.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=14708], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706], [object_number=13, tag_tid=1, entry_tid=14705]]], state=connected] 2024-04-23T21:50:21.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:21.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:21.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_48 2024-04-23T21:50:21.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:21.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:21.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:21.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_48 '..*' 2024-04-23T21:50:21.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:21.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:21.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:21.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:21.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_48.status 2024-04-23T21:50:21.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_48.status 2024-04-23T21:50:21.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:21.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:21.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_48 2024-04-23T21:50:21.113 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:21.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:21.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:21.154 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:21.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706]]], state=connected] 2024-04-23T21:50:21.155 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=14708], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706], [object_number=13, tag_tid=1, entry_tid=14705]]], state=connected] 2024-04-23T21:50:21.156 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:21.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=12, tag_tid=1, entry_tid=14708' 2024-04-23T21:50:21.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=12, tag_tid=1, entry_tid=14708' 2024-04-23T21:50:21.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=12, tag_tid=1, entry_tid=14708' '!=' '' 2024-04-23T21:50:21.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:50:21.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=12, tag_tid=1, entry_tid=14708' = '' 2024-04-23T21:50:21.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=12, tag_tid=1, entry_tid=14708' 2024-04-23T21:50:21.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:21.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:50:21.159 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:50:21.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=2 2024-04-23T21:50:21.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=12, tag_tid=1, entry_tid=14708' 2024-04-23T21:50:21.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:50:21.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:50:21.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=1 2024-04-23T21:50:21.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:21.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:50:21.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:50:21.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:50:21.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=12, tag_tid=1, entry_tid=14708' 2024-04-23T21:50:21.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:50:21.163 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:50:21.164 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=14708 2024-04-23T21:50:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 2 = 1 -a 1 -le 14708 2024-04-23T21:50:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:50:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:21.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:50:21.366 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_48 2024-04-23T21:50:21.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:21.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:21.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:50:21.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:21.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_48 ']' 2024-04-23T21:50:21.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_48' 2024-04-23T21:50:21.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_48 2024-04-23T21:50:21.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:21.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:21.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:21.367 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:21.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:21.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:21.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:21.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:21.368 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_48 2024-04-23T21:50:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:21.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:21.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:21.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:21.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:21.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:21.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:21.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_48 2024-04-23T21:50:21.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_48'] 2024-04-23T21:50:21.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:21.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:21.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:21.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_48 2024-04-23T21:50:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:21.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:21.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:21.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:21.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:21.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:21.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:21.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:21.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:21.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:21.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:21.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:21.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:21.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:21.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:21.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:21.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:21.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:21.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:21.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:21.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:21.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:21.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:21.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_48 2024-04-23T21:50:21.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:21.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_48 2024-04-23T21:50:21.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:21.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:21.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:21.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_48 '' 2024-04-23T21:50:21.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:21.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:21.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:21.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:21.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_48.status 2024-04-23T21:50:21.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_48.status 2024-04-23T21:50:21.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:21.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:21.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_48 2024-04-23T21:50:21.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:21.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:21.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:21.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:21.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706]]], state=connected] 2024-04-23T21:50:21.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=14708], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706], [object_number=13, tag_tid=1, entry_tid=14705]]], state=connected] 2024-04-23T21:50:21.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:21.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:21.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_48 2024-04-23T21:50:21.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:21.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:21.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:21.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_48 '..*' 2024-04-23T21:50:21.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:21.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:21.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:21.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:21.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_48.status 2024-04-23T21:50:21.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_48.status 2024-04-23T21:50:21.625 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:21.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:21.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_48 2024-04-23T21:50:21.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:21.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:21.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:21.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:21.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706]]], state=connected] 2024-04-23T21:50:21.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=14708], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706], [object_number=13, tag_tid=1, entry_tid=14705]]], state=connected] 2024-04-23T21:50:21.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:21.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=12, tag_tid=1, entry_tid=14708' 2024-04-23T21:50:21.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=12, tag_tid=1, entry_tid=14708' 2024-04-23T21:50:21.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=12, tag_tid=1, entry_tid=14708' '!=' 'object_number=12, tag_tid=1, entry_tid=14708' 2024-04-23T21:50:21.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:21.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:50:22.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_48 2024-04-23T21:50:22.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:22.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:50:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_48 ']' 2024-04-23T21:50:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_48' 2024-04-23T21:50:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_48 2024-04-23T21:50:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:22.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:22.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:22.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_48 2024-04-23T21:50:22.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:22.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:22.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:22.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:22.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:22.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:22.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:22.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:22.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:22.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:22.080 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:22.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:22.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:22.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:22.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:22.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:22.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:22.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:22.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:22.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:22.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:22.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_48 2024-04-23T21:50:22.175 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_48'] 2024-04-23T21:50:22.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:22.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:22.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:22.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_48 2024-04-23T21:50:22.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:22.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:22.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:22.179 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:22.180 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:22.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:22.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:22.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:22.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:22.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:22.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:22.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:22.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:22.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:22.196 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:22.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:22.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:22.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_48 2024-04-23T21:50:22.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:22.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_48 2024-04-23T21:50:22.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:22.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:22.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:22.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_48 '' 2024-04-23T21:50:22.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:22.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:22.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:22.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:22.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_48.status 2024-04-23T21:50:22.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_48.status 2024-04-23T21:50:22.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:22.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:22.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_48 2024-04-23T21:50:22.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:22.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:22.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:22.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:22.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706]]], state=connected] 2024-04-23T21:50:22.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=14708], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706], [object_number=13, tag_tid=1, entry_tid=14705]]], state=connected] 2024-04-23T21:50:22.330 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:22.331 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:22.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_48 2024-04-23T21:50:22.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:22.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:22.332 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:22.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_48 '..*' 2024-04-23T21:50:22.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:22.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:22.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:22.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:22.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_48.status 2024-04-23T21:50:22.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_48.status 2024-04-23T21:50:22.333 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:22.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:22.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_48 2024-04-23T21:50:22.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:22.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:22.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:22.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:22.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706]]], state=connected] 2024-04-23T21:50:22.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=14708], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706], [object_number=13, tag_tid=1, entry_tid=14705]]], state=connected] 2024-04-23T21:50:22.378 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:22.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=12, tag_tid=1, entry_tid=14708' 2024-04-23T21:50:22.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=12, tag_tid=1, entry_tid=14708' 2024-04-23T21:50:22.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=12, tag_tid=1, entry_tid=14708' '!=' 'object_number=12, tag_tid=1, entry_tid=14708' 2024-04-23T21:50:22.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:22.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:50:23.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_48 2024-04-23T21:50:23.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:23.181 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:23.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:50:23.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:23.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_48 ']' 2024-04-23T21:50:23.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_48' 2024-04-23T21:50:23.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_48 2024-04-23T21:50:23.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:23.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:23.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:23.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:23.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:23.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:23.182 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:23.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_48 2024-04-23T21:50:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:23.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:23.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:23.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:23.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:23.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:23.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:23.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:23.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:23.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:23.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:23.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:23.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:23.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:23.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:23.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:23.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:23.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:23.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:23.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:23.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:23.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:23.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_48 2024-04-23T21:50:23.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_48'] 2024-04-23T21:50:23.291 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:23.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:23.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:23.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_48 2024-04-23T21:50:23.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:23.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:23.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:23.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:23.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:23.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:23.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:23.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:23.296 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:23.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:23.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:23.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:23.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:23.312 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_48 2024-04-23T21:50:23.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:23.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_48 2024-04-23T21:50:23.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:23.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:23.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:23.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_48 '' 2024-04-23T21:50:23.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:23.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:23.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:23.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:23.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_48.status 2024-04-23T21:50:23.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_48.status 2024-04-23T21:50:23.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:23.405 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:23.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_48 2024-04-23T21:50:23.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:23.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:23.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:23.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:23.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706]]], state=connected] 2024-04-23T21:50:23.447 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=14708], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706], [object_number=13, tag_tid=1, entry_tid=14705]]], state=connected] 2024-04-23T21:50:23.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:23.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:23.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_48 2024-04-23T21:50:23.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:23.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:23.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:23.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_48 '..*' 2024-04-23T21:50:23.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:23.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:23.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:23.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:23.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_48.status 2024-04-23T21:50:23.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_48.status 2024-04-23T21:50:23.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:23.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:23.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_48 2024-04-23T21:50:23.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:23.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:23.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:23.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:23.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706]]], state=connected] 2024-04-23T21:50:23.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=12, tag_tid=1, entry_tid=14708], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706], [object_number=13, tag_tid=1, entry_tid=14705]]], state=connected] 2024-04-23T21:50:23.498 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:23.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=12, tag_tid=1, entry_tid=14708' 2024-04-23T21:50:23.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=12, tag_tid=1, entry_tid=14708' 2024-04-23T21:50:23.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=12, tag_tid=1, entry_tid=14708' '!=' 'object_number=12, tag_tid=1, entry_tid=14708' 2024-04-23T21:50:23.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:23.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:50:25.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_48 2024-04-23T21:50:25.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:25.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:25.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:50:25.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:25.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_48 ']' 2024-04-23T21:50:25.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_48' 2024-04-23T21:50:25.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_48 2024-04-23T21:50:25.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:25.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:25.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:25.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:25.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:25.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:25.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:25.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:25.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:25.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:25.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_48 2024-04-23T21:50:25.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:25.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:25.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:25.106 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:25.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:25.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:25.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:25.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:25.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_48 2024-04-23T21:50:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_48'] 2024-04-23T21:50:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:25.211 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:25.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:25.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_48 2024-04-23T21:50:25.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:25.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:25.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:25.214 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:25.215 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:25.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:25.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:25.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:25.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:25.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:25.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:25.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:25.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:25.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:25.216 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:25.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:25.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:25.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:25.233 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_48 2024-04-23T21:50:25.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:25.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_48 2024-04-23T21:50:25.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:25.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:25.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:25.326 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_48 '' 2024-04-23T21:50:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_48.status 2024-04-23T21:50:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_48.status 2024-04-23T21:50:25.327 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:25.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:25.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_48 2024-04-23T21:50:25.328 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:25.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:25.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:25.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:25.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706]]], state=connected] 2024-04-23T21:50:25.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706]]], state=connected] 2024-04-23T21:50:25.371 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:25.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_48 2024-04-23T21:50:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_48 '..*' 2024-04-23T21:50:25.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:25.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:25.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:50:25.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:25.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_48.status 2024-04-23T21:50:25.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_48.status 2024-04-23T21:50:25.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:25.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:25.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_48 2024-04-23T21:50:25.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 3 2024-04-23T21:50:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 3 2024-04-23T21:50:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:25.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706]]], state=connected] 2024-04-23T21:50:25.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=13, tag_tid=2, entry_tid=1], [object_number=12, tag_tid=2, entry_tid=0], [object_number=15, tag_tid=1, entry_tid=14707], [object_number=14, tag_tid=1, entry_tid=14706]]], state=connected] 2024-04-23T21:50:25.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.status 2024-04-23T21:50:25.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:25.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=13, tag_tid=2, entry_tid=1' -a 'object_number=13, tag_tid=2, entry_tid=1' = 'object_number=13, tag_tid=2, entry_tid=1' 2024-04-23T21:50:25.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:25.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_48 snap 2024-04-23T21:50:25.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:25.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:25.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:50:25.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:50:25.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:50:25.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:50:25.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:50:26.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_48@snap 2024-04-23T21:50:26.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_48': 2024-04-23T21:50:26.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:50:26.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:50:26.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:50:26.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 12326e351a3 2024-04-23T21:50:26.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.12326e351a3 2024-04-23T21:50:26.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:50:26.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:50:26.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:50:26.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:50:26.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:39:42 2024 2024-04-23T21:50:26.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:39:42 2024 2024-04-23T21:50:26.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:40:43 2024 2024-04-23T21:50:26.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:50:26.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 12326e351a3 2024-04-23T21:50:26.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:50:26.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:50:26.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 2036884a-00e7-4ea6-ab44-13b4c29e39c1 2024-04-23T21:50:26.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:50:26.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:26.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_48 snap 2024-04-23T21:50:26.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:26.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:50:26.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:50:26.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:50:26.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.export 2024-04-23T21:50:26.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_48.export 2024-04-23T21:50:26.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_48.export 2024-04-23T21:50:26.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_48@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.export 2024-04-23T21:50:26.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:50:26.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_48@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_48.export 2024-04-23T21:50:26.902 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:50:26.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_48.export 2024-04-23T21:50:26.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_48.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_48.export 2024-04-23T21:50:26.998 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:26.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:26.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_49 2024-04-23T21:50:26.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_49 snap 2024-04-23T21:50:26.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:26.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:26.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:50:26.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:50:26.999 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_49@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:50:28.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_49 2024-04-23T21:50:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:50:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_49 started 2024-04-23T21:50:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:50:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:50:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:50:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:28.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:50:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_49 started 2024-04-23T21:50:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:50:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:50:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:50:29.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:50:29.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_49 2024-04-23T21:50:29.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:50:29.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:50:29.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:50:29.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:29.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:50:29.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:50:29.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:29.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:50:29.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:29.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:50:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_49 2024-04-23T21:50:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:50:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:50:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:29.430 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:29.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:29.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:50:29.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:29.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:29.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:29.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:29.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:50:29.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:29.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:29.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:29.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:29.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:50:29.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:50:29.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:50:29.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:29.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:50:29.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:50:29.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:50:29.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:29.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:29.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:29.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:50:29.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:50:29.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:50:29.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:50:29.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:29.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:29.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:29.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:29.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_49 2024-04-23T21:50:29.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_49'] 2024-04-23T21:50:29.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:50:29.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_49 2024-04-23T21:50:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:50:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:50:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:50:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:29.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:50:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:50:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:50:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:50:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:50:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:50:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:50:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:50:29.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:50:29.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:50:29.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:50:29.549 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:29.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:29.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:29.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:29.565 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_49 2024-04-23T21:50:29.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:50:29.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:50:29.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:50:29.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:50:29.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:50:29.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:50:29.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:50:29.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:29.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_49 2024-04-23T21:50:29.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:50:29.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:29.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:29.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:50:29.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:50:29.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_49 2024-04-23T21:50:29.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:50:29.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:29.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:29.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:50:29.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:50:29.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:50:29.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:50:29.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:29.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:50:29.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_49 2024-04-23T21:50:29.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:29.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:29.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:50:29.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:29.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_49 ']' 2024-04-23T21:50:29.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_49' 2024-04-23T21:50:29.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_49 2024-04-23T21:50:29.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:29.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:29.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:29.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:29.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:29.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:29.862 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_49 2024-04-23T21:50:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:29.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:29.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:29.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:29.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:29.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:29.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:29.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:29.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:29.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:29.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:29.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:29.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:29.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:29.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:29.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:29.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:29.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:29.868 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:29.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:29.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:29.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:29.882 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_49 2024-04-23T21:50:29.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_49'] 2024-04-23T21:50:29.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:29.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:29.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:29.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_49 2024-04-23T21:50:29.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:29.981 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:29.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:29.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:29.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:29.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:29.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:29.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:29.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:29.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:29.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:29.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:29.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:29.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:29.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:29.982 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:29.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:29.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:29.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:29.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:29.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:29.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:29.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:29.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:29.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:29.983 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:29.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:29.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:29.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:29.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:29.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:29.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:29.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:29.985 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_49 2024-04-23T21:50:30.075 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:30.076 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_49 2024-04-23T21:50:30.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:30.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:30.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_49 2024-04-23T21:50:30.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_49 '' 2024-04-23T21:50:30.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:30.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:30.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_49 2024-04-23T21:50:30.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:30.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_49.status 2024-04-23T21:50:30.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_49.status 2024-04-23T21:50:30.077 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:30.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_49.status 2024-04-23T21:50:30.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_49 2024-04-23T21:50:30.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_49.status 2024-04-23T21:50:30.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:50:30.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:50:30.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:30.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10283], [object_number=10, tag_tid=1, entry_tid=10282]]], state=connected] 2024-04-23T21:50:30.119 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10283], [object_number=10, tag_tid=1, entry_tid=10282]]], state=connected] 2024-04-23T21:50:30.122 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_49.status 2024-04-23T21:50:30.123 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:50:30.124 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_49 2024-04-23T21:50:30.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:30.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:30.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_49 2024-04-23T21:50:30.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_49 '..*' 2024-04-23T21:50:30.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:30.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:30.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_49 2024-04-23T21:50:30.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:30.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_49.status 2024-04-23T21:50:30.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_49.status 2024-04-23T21:50:30.125 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:30.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_49.status 2024-04-23T21:50:30.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_49 2024-04-23T21:50:30.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_49.status 2024-04-23T21:50:30.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 2 2024-04-23T21:50:30.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 2 2024-04-23T21:50:30.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:30.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10283], [object_number=10, tag_tid=1, entry_tid=10282]]], state=connected] 2024-04-23T21:50:30.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=9, tag_tid=2, entry_tid=1], [object_number=8, tag_tid=2, entry_tid=0], [object_number=11, tag_tid=1, entry_tid=10283], [object_number=10, tag_tid=1, entry_tid=10282]]], state=connected] 2024-04-23T21:50:30.170 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_49.status 2024-04-23T21:50:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:50:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=9, tag_tid=2, entry_tid=1' -a 'object_number=9, tag_tid=2, entry_tid=1' = 'object_number=9, tag_tid=2, entry_tid=1' 2024-04-23T21:50:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:30.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_49 snap 2024-04-23T21:50:30.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:30.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:30.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:50:30.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:50:30.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:50:30.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:50:30.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:50:31.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_49@snap 2024-04-23T21:50:31.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_49': 2024-04-23T21:50:31.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:50:31.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:50:31.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:50:31.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 116e1aca8631 2024-04-23T21:50:31.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.116e1aca8631 2024-04-23T21:50:31.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:50:31.221 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:50:31.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:50:31.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:50:31.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:39:58 2024 2024-04-23T21:50:31.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:39:58 2024 2024-04-23T21:50:31.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:40:59 2024 2024-04-23T21:50:31.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:50:31.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 116e1aca8631 2024-04-23T21:50:31.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:50:31.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:50:31.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: b76b9655-fc98-4703-a24f-97920c971f0d 2024-04-23T21:50:31.222 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:50:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_49 snap 2024-04-23T21:50:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:50:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:50:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:50:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_49.export 2024-04-23T21:50:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_49.export 2024-04-23T21:50:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_49.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_49.export 2024-04-23T21:50:31.225 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_49@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_49.export 2024-04-23T21:50:31.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:50:31.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_49@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_49.export 2024-04-23T21:50:31.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:50:31.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_49.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_49.export 2024-04-23T21:50:31.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_49.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_49.export 2024-04-23T21:50:31.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:31.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:31.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_50 2024-04-23T21:50:31.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_snap cluster2 mirror image_50 snap 2024-04-23T21:50:31.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:31.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:31.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:50:31.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:50:31.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror snap create image_50@snap --debug-rbd=20 --debug-journaler=20 2024-04-23T21:50:32.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_started cluster1 mirror image_50 2024-04-23T21:50:32.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:32.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:32.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:50:32.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_replay_state cluster1 mirror image_50 started 2024-04-23T21:50:32.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:32.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:32.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:50:32.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=started 2024-04-23T21:50:32.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:50:32.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:32.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:50:33.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_replay_state cluster1 mirror image_50 started 2024-04-23T21:50:33.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:33.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:33.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:50:33.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=started 2024-04-23T21:50:33.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local status_result 2024-04-23T21:50:33.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=stopped 2024-04-23T21:50:33.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_50 2024-04-23T21:50:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_instance=cluster1 2024-04-23T21:50:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -i state 2024-04-23T21:50:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:50:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance=cluster1 2024-04-23T21:50:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local loop_instance 2024-04-23T21:50:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sleep 0 2024-04-23T21:50:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:33.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ seq 0 3 2024-04-23T21:50:33.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:33.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_50 2024-04-23T21:50:33.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:0 2024-04-23T21:50:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:50:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:50:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:50:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:0 2024-04-23T21:50:33.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:50:33.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:50:33.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:33.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:0 2024-04-23T21:50:33.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:50:33.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:50:33.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:33.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:50:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:50:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=0 2024-04-23T21:50:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=0 2024-04-23T21:50:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:33.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:33.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:33.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:33.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_50 2024-04-23T21:50:33.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_50'] 2024-04-23T21:50:33.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:50:33.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:33.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:33.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_50 2024-04-23T21:50:33.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:1 2024-04-23T21:50:33.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:50:33.551 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:33.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:33.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:33.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:50:33.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:33.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:33.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:33.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:33.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:50:33.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:33.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:33.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:33.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:33.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:1 2024-04-23T21:50:33.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:50:33.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:50:33.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:33.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:1 2024-04-23T21:50:33.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:50:33.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:50:33.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:33.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:33.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:33.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:50:33.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:50:33.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=1 2024-04-23T21:50:33.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=1 2024-04-23T21:50:33.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:33.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:33.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:33.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:33.569 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_50 2024-04-23T21:50:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_50'] 2024-04-23T21:50:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror status : get status for rbd mirror 2024-04-23T21:50:33.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_50 2024-04-23T21:50:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1:2 2024-04-23T21:50:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ shift 2024-04-23T21:50:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:33.658 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:50:33.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:50:33.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=cluster 2024-04-23T21:50:33.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:33.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:33.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:50:33.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:50:33.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval cluster=cluster1 2024-04-23T21:50:33.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:33.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:50:33.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:50:33.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:50:33.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local local_cluster=cluster1:2 2024-04-23T21:50:33.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster=cluster1 2024-04-23T21:50:33.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance 2024-04-23T21:50:33.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:50:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local val=cluster1:2 2024-04-23T21:50:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local cluster_var_name=local_cluster 2024-04-23T21:50:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local instance_var_name=instance 2024-04-23T21:50:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ cluster=cluster1 2024-04-23T21:50:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:50:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ '[' 2 = cluster1:2 ']' 2024-04-23T21:50:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval local_cluster=cluster1 2024-04-23T21:50:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ local_cluster=cluster1 2024-04-23T21:50:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ eval instance=2 2024-04-23T21:50:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ instance=2 2024-04-23T21:50:33.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:50:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:33.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_50 2024-04-23T21:50:33.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ return 0 2024-04-23T21:50:33.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ status_result=' "state": "Replaying"' 2024-04-23T21:50:33.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo ' "state": "Replaying"' 2024-04-23T21:50:33.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ grep -i Replaying 2024-04-23T21:50:33.770 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: "state": "Replaying" 2024-04-23T21:50:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=started 2024-04-23T21:50:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test started = started 2024-04-23T21:50:33.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror image_50 2024-04-23T21:50:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:50:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:50:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' journal = journal ']' 2024-04-23T21:50:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_journal_replay_complete cluster1 cluster2 mirror image_50 2024-04-23T21:50:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local local_cluster=cluster1 2024-04-23T21:50:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:50:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s master_pos mirror_pos last_mirror_pos 2024-04-23T21:50:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local master_tag master_entry mirror_tag mirror_entry 2024-04-23T21:50:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:50:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:33.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:50:33.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_50 2024-04-23T21:50:33.972 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:50:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_50 ']' 2024-04-23T21:50:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_50' 2024-04-23T21:50:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_50 2024-04-23T21:50:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:33.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:33.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_50 2024-04-23T21:50:33.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:33.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:33.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:33.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:33.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:33.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:33.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:33.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:33.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:33.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:33.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:33.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:33.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:33.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:33.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:33.978 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:33.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:33.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:34.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_50'] 2024-04-23T21:50:34.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:34.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:34.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_50 2024-04-23T21:50:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:34.085 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:34.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:34.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:34.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:34.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:34.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:34.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:34.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:34.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:34.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_50'] 2024-04-23T21:50:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:34.183 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_50 2024-04-23T21:50:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:50:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:50:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:50:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:50:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:50:34.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:50:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:50:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:50:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:50:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:50:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:50:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:50:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:50:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:34.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:50:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:50:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:50:34.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:34.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:34.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:34.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:34.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:34.292 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_50 2024-04-23T21:50:34.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:34.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:34.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:34.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_50 '' 2024-04-23T21:50:34.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:34.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:34.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:34.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:34.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_50.status 2024-04-23T21:50:34.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_50.status 2024-04-23T21:50:34.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:34.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:34.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_50 2024-04-23T21:50:34.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:34.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:34.335 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:34.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:34.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:50:34.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_50 2024-04-23T21:50:34.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:34.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:34.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:34.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_50 '..*' 2024-04-23T21:50:34.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:34.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:34.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:34.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:34.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_50.status 2024-04-23T21:50:34.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_50.status 2024-04-23T21:50:34.340 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:34.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:34.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_50 2024-04-23T21:50:34.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:34.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:34.385 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:34.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:34.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:34.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' '' 2024-04-23T21:50:34.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ break 2024-04-23T21:50:34.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' = '' 2024-04-23T21:50:34.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ last_mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:34.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:50:34.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:50:34.387 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:50:34.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_tag=3 2024-04-23T21:50:34.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:34.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'tag_tid=[0-9]*' 2024-04-23T21:50:34.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:50:34.389 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_tag=2 2024-04-23T21:50:34.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:50:34.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:50:34.390 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:50:34.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_entry=1 2024-04-23T21:50:34.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:34.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -Eo 'entry_tid=[0-9]*' 2024-04-23T21:50:34.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut -d= -f 2 2024-04-23T21:50:34.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_entry=1 2024-04-23T21:50:34.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 3 = 2 -a 1 -le 1 2024-04-23T21:50:34.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:50:34.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:34.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.2 2024-04-23T21:50:34.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_50 2024-04-23T21:50:34.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:34.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:34.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:50:34.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:34.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_50 ']' 2024-04-23T21:50:34.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_50' 2024-04-23T21:50:34.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_50 2024-04-23T21:50:34.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:34.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:34.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:34.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:34.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:34.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:34.595 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:34.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:34.596 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:34.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:34.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_50 2024-04-23T21:50:34.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:34.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:34.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:34.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:34.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:34.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:34.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:34.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:34.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:34.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:34.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:34.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:34.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:34.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:34.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:34.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:34.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:34.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:34.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:34.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:34.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:34.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:34.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:34.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:34.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:34.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:34.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:34.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:34.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:34.600 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:34.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:34.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:34.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:34.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:34.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_50'] 2024-04-23T21:50:34.694 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:34.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:34.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:34.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_50 2024-04-23T21:50:34.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:34.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:34.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:34.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:34.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:34.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:34.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:34.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:34.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:34.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:34.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:34.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:34.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:34.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:34.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:34.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:34.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:34.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:34.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:34.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:34.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:34.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:34.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:34.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:34.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:34.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:34.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:34.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:34.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:34.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:34.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:34.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:34.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:34.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:34.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_50'] 2024-04-23T21:50:34.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:34.797 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:34.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:34.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_50 2024-04-23T21:50:34.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:50:34.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:34.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:34.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:50:34.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:50:34.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:34.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:34.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:34.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:50:34.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:50:34.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:34.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:34.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:50:34.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:50:34.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:50:34.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:50:34.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:34.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:34.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:50:34.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:50:34.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:34.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:34.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:34.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:50:34.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:50:34.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:34.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:34.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:50:34.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:50:34.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:50:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:34.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:34.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_50 2024-04-23T21:50:34.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:34.910 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:34.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:34.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_50 '' 2024-04-23T21:50:34.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:34.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:34.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:34.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:34.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_50.status 2024-04-23T21:50:34.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_50.status 2024-04-23T21:50:34.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:34.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:34.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_50 2024-04-23T21:50:34.912 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:34.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:34.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:34.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:50:34.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_50 2024-04-23T21:50:34.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:34.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:34.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:34.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_50 '..*' 2024-04-23T21:50:34.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:34.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:34.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:34.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:34.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_50.status 2024-04-23T21:50:34.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_50.status 2024-04-23T21:50:34.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:34.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:34.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_50 2024-04-23T21:50:34.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:35.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:35.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:35.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:35.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:35.003 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:35.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:35.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:35.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:35.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:35.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:35.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.4 2024-04-23T21:50:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_50 2024-04-23T21:50:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:50:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_50 ']' 2024-04-23T21:50:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_50' 2024-04-23T21:50:35.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_50 2024-04-23T21:50:35.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:35.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:35.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:35.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:35.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:35.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:35.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:35.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:35.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:35.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:35.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_50 2024-04-23T21:50:35.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:35.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:35.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:35.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:35.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:35.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:35.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:35.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:35.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:35.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:35.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:35.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:35.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:35.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:35.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:35.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:35.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:35.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:35.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:35.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:35.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_50'] 2024-04-23T21:50:35.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:35.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:35.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:35.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_50 2024-04-23T21:50:35.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:35.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:35.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:35.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:35.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:35.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:35.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:35.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:35.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:35.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:35.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:35.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:35.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:35.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:35.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:35.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:35.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:35.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:35.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:35.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:35.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:35.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:35.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:35.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:35.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:35.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:35.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:35.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:35.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:35.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:35.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:35.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:35.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:35.532 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:35.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_50'] 2024-04-23T21:50:35.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:35.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:35.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_50 2024-04-23T21:50:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:50:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:50:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:50:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:50:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:50:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:35.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:50:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:50:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:50:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:50:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:50:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:50:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:50:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:50:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:35.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:35.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:50:35.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:50:35.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:50:35.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:35.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:35.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:35.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:35.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_50 2024-04-23T21:50:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_50 '' 2024-04-23T21:50:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:35.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_50.status 2024-04-23T21:50:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_50.status 2024-04-23T21:50:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_50 2024-04-23T21:50:35.724 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:35.766 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:35.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:35.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:35.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:35.767 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:35.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:35.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:50:35.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_50 2024-04-23T21:50:35.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:35.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:35.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:35.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_50 '..*' 2024-04-23T21:50:35.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:35.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:35.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:35.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:35.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_50.status 2024-04-23T21:50:35.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_50.status 2024-04-23T21:50:35.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:35.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:35.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_50 2024-04-23T21:50:35.773 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:35.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:35.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:35.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:35.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:35.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:35.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:35.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:35.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:35.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:35.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:35.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.8 2024-04-23T21:50:36.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_50 2024-04-23T21:50:36.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:36.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:36.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:50:36.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:36.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_50 ']' 2024-04-23T21:50:36.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_50' 2024-04-23T21:50:36.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_50 2024-04-23T21:50:36.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:36.622 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:36.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:36.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:36.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:36.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:36.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:36.623 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_50 2024-04-23T21:50:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:36.626 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:36.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:36.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:36.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:36.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:36.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:36.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:36.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:36.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_50'] 2024-04-23T21:50:36.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:36.735 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:36.738 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:36.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_50 2024-04-23T21:50:36.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:36.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:36.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:36.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:36.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:36.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:36.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:36.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:36.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:36.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:36.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:36.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:36.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:36.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:36.739 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:36.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:36.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:36.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:36.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:36.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:36.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:36.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:36.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:36.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:36.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:36.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:36.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:36.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:36.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:36.740 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:36.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:36.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:36.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:36.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:36.836 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_50'] 2024-04-23T21:50:36.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:36.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:36.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:36.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_50 2024-04-23T21:50:36.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:50:36.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:36.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:36.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:50:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:50:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:50:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:50:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:50:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:50:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:50:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:50:36.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:50:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:50:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:50:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:50:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:50:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:50:36.842 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:50:36.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:36.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:36.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:36.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:36.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:36.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_50 2024-04-23T21:50:36.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:36.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:36.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:36.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_50 '' 2024-04-23T21:50:36.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:36.945 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:36.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:36.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:36.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_50.status 2024-04-23T21:50:36.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_50.status 2024-04-23T21:50:36.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:36.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:36.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_50 2024-04-23T21:50:36.947 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:36.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:36.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:36.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:36.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:36.989 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:36.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:36.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:50:36.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_50 2024-04-23T21:50:36.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:36.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:36.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_50 '..*' 2024-04-23T21:50:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_50.status 2024-04-23T21:50:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_50.status 2024-04-23T21:50:36.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_50 2024-04-23T21:50:36.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:37.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:37.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:37.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:37.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:37.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:37.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:37.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:37.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:37.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:37.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:37.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1.6 2024-04-23T21:50:38.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_50 2024-04-23T21:50:38.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:38.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:38.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:50:38.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:38.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_50 ']' 2024-04-23T21:50:38.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_50' 2024-04-23T21:50:38.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_50 2024-04-23T21:50:38.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:38.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:38.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:38.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:38.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:38.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:38.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:38.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:38.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:38.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:38.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_50 2024-04-23T21:50:38.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:38.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:38.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:38.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:38.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:38.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:38.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:38.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:38.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:38.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:38.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:38.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:38.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:38.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:38.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:38.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:38.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:38.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:38.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:38.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:38.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:38.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:38.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:38.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:38.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:38.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:38.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:38.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:38.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:38.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:38.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:38.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:38.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:38.668 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:38.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_50'] 2024-04-23T21:50:38.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:38.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:38.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:38.757 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_50 2024-04-23T21:50:38.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:38.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:38.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:38.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:38.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:38.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:38.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:38.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:38.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:38.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:38.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:38.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:38.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:38.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:38.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:38.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:38.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:38.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:38.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:38.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:38.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:38.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_50'] 2024-04-23T21:50:38.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:38.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:38.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:38.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_50 2024-04-23T21:50:38.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:50:38.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:38.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:38.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:50:38.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:50:38.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:38.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:38.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:38.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:50:38.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:50:38.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:38.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:38.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:50:38.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:50:38.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:50:38.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:50:38.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:38.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:38.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:50:38.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:50:38.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:38.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:38.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:38.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:50:38.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:50:38.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:38.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:38.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:50:38.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:50:38.861 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:50:38.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:38.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:38.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:38.877 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:38.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:38.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_50 2024-04-23T21:50:38.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:38.963 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:38.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:38.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_50 '' 2024-04-23T21:50:38.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:38.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:38.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:38.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:38.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_50.status 2024-04-23T21:50:38.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_50.status 2024-04-23T21:50:38.964 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:38.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:38.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_50 2024-04-23T21:50:38.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:39.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:39.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:39.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:39.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:39.006 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:39.008 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:39.010 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:50:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_50 2024-04-23T21:50:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_50 '..*' 2024-04-23T21:50:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:39.011 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_50.status 2024-04-23T21:50:39.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_50.status 2024-04-23T21:50:39.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:39.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:39.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_50 2024-04-23T21:50:39.012 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:39.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:39.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:39.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:39.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:39.054 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=2, entry_tid=1], [object_number=4, tag_tid=2, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:39.057 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:39.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:39.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:39.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 'object_number=5, tag_tid=2, entry_tid=1' '!=' 'object_number=5, tag_tid=2, entry_tid=1' 2024-04-23T21:50:39.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2024-04-23T21:50:39.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:50:41.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ flush cluster1 mirror image_50 2024-04-23T21:50:41.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:41.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:41.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:50:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local 'cmd=rbd mirror flush' 2024-04-23T21:50:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n image_50 ']' 2024-04-23T21:50:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmd='rbd mirror flush mirror/image_50' 2024-04-23T21:50:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemons cluster1 rbd mirror flush mirror/image_50 2024-04-23T21:50:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_instance=cluster1 2024-04-23T21:50:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance=cluster1 2024-04-23T21:50:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loop_instance 2024-04-23T21:50:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2024-04-23T21:50:41.061 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 '!=' cluster1 ']' 2024-04-23T21:50:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:50:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:0 rbd mirror flush mirror/image_50 2024-04-23T21:50:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:50:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:0 cluster instance 2024-04-23T21:50:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:0 2024-04-23T21:50:41.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:50:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:50:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:0 cluster1 2024-04-23T21:50:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:0 2024-04-23T21:50:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:0 local_cluster instance 2024-04-23T21:50:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:0 2024-04-23T21:50:41.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:41.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:41.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:41.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:50:41.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 0 = cluster1:0 ']' 2024-04-23T21:50:41.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:41.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:41.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=0 2024-04-23T21:50:41.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=0 2024-04-23T21:50:41.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2024-04-23T21:50:41.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:41.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:41.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2024-04-23T21:50:41.084 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:41.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_50'] 2024-04-23T21:50:41.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:41.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:41.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:41.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:1 rbd mirror flush mirror/image_50 2024-04-23T21:50:41.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:50:41.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:41.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:41.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:1 cluster instance 2024-04-23T21:50:41.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:1 2024-04-23T21:50:41.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:41.171 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:41.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:41.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=1 2024-04-23T21:50:41.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:41.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:41.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:41.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=1 2024-04-23T21:50:41.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:41.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:1 cluster1 2024-04-23T21:50:41.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:1 2024-04-23T21:50:41.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:41.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:41.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:1 local_cluster instance 2024-04-23T21:50:41.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:1 2024-04-23T21:50:41.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:41.172 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:41.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:41.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=1 2024-04-23T21:50:41.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 1 = cluster1:1 ']' 2024-04-23T21:50:41.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:41.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:41.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=1 2024-04-23T21:50:41.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=1 2024-04-23T21:50:41.173 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2024-04-23T21:50:41.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:41.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:41.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2024-04-23T21:50:41.188 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:41.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:Invalid command: unused arguments: ['mirror/image_50'] 2024-04-23T21:50:41.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd mirror flush : flush rbd mirror 2024-04-23T21:50:41.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:admin_socket: invalid command 2024-04-23T21:50:41.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:50:41.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ admin_daemon cluster1:2 rbd mirror flush mirror/image_50 2024-04-23T21:50:41.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:50:41.271 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:50:41.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:50:41.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1:2 cluster instance 2024-04-23T21:50:41.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1:2 2024-04-23T21:50:41.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:50:41.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:50:41.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:50:41.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=2 2024-04-23T21:50:41.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 2 = cluster1:2 ']' 2024-04-23T21:50:41.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:50:41.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:41.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=2 2024-04-23T21:50:41.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:50:41.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ daemon_asok_file cluster1:2 cluster1 2024-04-23T21:50:41.272 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local local_cluster=cluster1:2 2024-04-23T21:50:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:50:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance 2024-04-23T21:50:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ set_cluster_instance cluster1:2 local_cluster instance 2024-04-23T21:50:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local val=cluster1:2 2024-04-23T21:50:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster_var_name=local_cluster 2024-04-23T21:50:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local instance_var_name=instance 2024-04-23T21:50:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:50:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=2 2024-04-23T21:50:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ '[' 2 = cluster1:2 ']' 2024-04-23T21:50:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval local_cluster=cluster1 2024-04-23T21:50:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ local_cluster=cluster1 2024-04-23T21:50:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ eval instance=2 2024-04-23T21:50:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ instance=2 2024-04-23T21:50:41.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2024-04-23T21:50:41.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:41.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:41.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2024-04-23T21:50:41.289 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror flush mirror/image_50 2024-04-23T21:50:41.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:41.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_master_journal_position cluster2 mirror image_50 2024-04-23T21:50:41.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:41.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:41.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:41.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_50 '' 2024-04-23T21:50:41.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:41.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:41.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:41.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local id_regexp= 2024-04-23T21:50:41.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_50.status 2024-04-23T21:50:41.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_50.status 2024-04-23T21:50:41.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:41.383 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:41.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_50 2024-04-23T21:50:41.384 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:41.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:41.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:41.425 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:41.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:41.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:41.428 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:41.429 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ master_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:50:41.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_mirror_journal_position cluster2 mirror image_50 2024-04-23T21:50:41.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:41.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:41.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:41.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_journal_position cluster2 mirror image_50 '..*' 2024-04-23T21:50:41.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:50:41.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:50:41.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:50:41.431 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local 'id_regexp=..*' 2024-04-23T21:50:41.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ mkfname cluster2-mirror-image_50.status 2024-04-23T21:50:41.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ echo cluster2-mirror-image_50.status 2024-04-23T21:50:41.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+++ sed -e 's|[/ ]|_|g' 2024-04-23T21:50:41.432 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local status_log=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:41.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster2 journal status --image mirror/image_50 2024-04-23T21:50:41.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ tee /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:41.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:minimum_set: 1 2024-04-23T21:50:41.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:active_set: 1 2024-04-23T21:50:41.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:registered clients: 2024-04-23T21:50:41.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:41.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: [id=4634d4db-65b6-4cfb-bbd3-aa8c5a218a39, commit_position=[positions=[[object_number=5, tag_tid=3, entry_tid=1], [object_number=4, tag_tid=3, entry_tid=0], [object_number=6, tag_tid=1, entry_tid=8318], [object_number=7, tag_tid=1, entry_tid=8315]]], state=connected] 2024-04-23T21:50:41.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -nEe 's/^.*\[id=..*,.*positions=\[\[([^]]*)\],.*state=connected.*$/\1/p' /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.status 2024-04-23T21:50:41.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ mirror_pos='object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:50:41.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 'object_number=5, tag_tid=3, entry_tid=1' -a 'object_number=5, tag_tid=3, entry_tid=1' = 'object_number=5, tag_tid=3, entry_tid=1' 2024-04-23T21:50:41.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:41.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_snap_present cluster1 mirror image_50 snap 2024-04-23T21:50:41.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:50:41.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:41.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:50:41.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:50:41.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:50:41.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2024-04-23T21:50:41.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:50:42.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 info mirror/image_50@snap 2024-04-23T21:50:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_50': 2024-04-23T21:50:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 128 MiB in 32 objects 2024-04-23T21:50:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:50:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 1 2024-04-23T21:50:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 11b8d3c7c7a3 2024-04-23T21:50:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.11b8d3c7c7a3 2024-04-23T21:50:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:50:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:50:42.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:50:42.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:50:42.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:40:08 2024 2024-04-23T21:50:42.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:40:08 2024 2024-04-23T21:50:42.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:41:09 2024 2024-04-23T21:50:42.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: protected: False 2024-04-23T21:50:42.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 11b8d3c7c7a3 2024-04-23T21:50:42.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:50:42.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:50:42.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 1c2e4006-7852-41f2-8c5f-4014ca5bdc58 2024-04-23T21:50:42.527 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: false 2024-04-23T21:50:42.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:42.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ compare_image_snaps mirror image_50 snap 2024-04-23T21:50:42.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:42.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:50:42.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap_name=snap 2024-04-23T21:50:42.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local ret=0 2024-04-23T21:50:42.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local rmt_export=/tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.export 2024-04-23T21:50:42.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local loc_export=/tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_50.export 2024-04-23T21:50:42.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_50.export 2024-04-23T21:50:42.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 -p mirror export image_50@snap /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.export 2024-04-23T21:50:42.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:50:42.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster1 -p mirror export image_50@snap /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_50.export 2024-04-23T21:50:42.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2024-04-23T21:50:42.942 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cmp /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_50.export 2024-04-23T21:50:42.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -f /tmp/tmp.8Tyc9aSFBO/cluster2-mirror-image_50.export /tmp/tmp.8Tyc9aSFBO/cluster1-mirror-image_50.export 2024-04-23T21:50:43.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:43.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ testlog 'TEST: delete many images' 2024-04-23T21:50:43.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ tee -a /tmp/tmp.8Tyc9aSFBO/rbd-mirror.test.log 2024-04-23T21:50:43.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ date '+%F %T' 2024-04-23T21:50:43.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo 2024-04-23 21:50:43 TEST: delete many images 2024-04-23T21:50:43.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:2024-04-23 21:50:43 TEST: delete many images 2024-04-23T21:50:43.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 1 50 2024-04-23T21:50:43.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:43.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_1 2024-04-23T21:50:43.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_1 snap 2024-04-23T21:50:43.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:43.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:43.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:50:43.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:43.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_1@snap 2024-04-23T21:50:43.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9831440644751219) > weight (0.25) 2024-04-23T21:50:43.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.60333739643384) > weight (0.25) 2024-04-23T21:50:43.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9051456651623098) > weight (0.25) 2024-04-23T21:50:43.306 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2024-04-23T21:50:43.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2024-04-23T21:50:43.307 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2024-04-23T21:50:43.307 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2024-04-23T21:50:43.307 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:50:43.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:50:43.308+0000 7f4390364640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 62958) UID: 1000 2024-04-23T21:50:43.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_1 2024-04-23T21:50:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:43.471 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:43.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:50:43.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:43.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:43.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_1 2024-04-23T21:50:43.472 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:43.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:43.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:50:43.473 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_1 2024-04-23T21:50:43.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_1 2024-04-23T21:50:43.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:43.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:43.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:43.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_2 2024-04-23T21:50:43.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_2 snap 2024-04-23T21:50:43.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:43.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:43.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:50:43.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:43.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_2@snap 2024-04-23T21:50:44.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:44.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_2 2024-04-23T21:50:44.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:44.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:44.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:50:44.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:44.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:44.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_2 2024-04-23T21:50:44.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:44.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:44.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:50:44.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_2 2024-04-23T21:50:44.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_2 2024-04-23T21:50:44.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:50:44.775+0000 7f9878494640 -1 rbd::mirror::image_replayer::journal::PrepareReplayRequest: 0x555f65654140 handle_get_remote_tags: failed to retrieve remote tags: (2) No such file or directory 2024-04-23T21:50:44.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:50:44.775+0000 7f9878494640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x555f65089a00 handle_prepare_replay: failed to prepare local replay: (2) No such file or directory 2024-04-23T21:50:44.921 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:50:44.921+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b3953ba0 2024-04-23T21:50:44.923 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:50:44.922+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b3957110 2024-04-23T21:50:44.923 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:50:44.922+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b54881a0 2024-04-23T21:50:44.924 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:50:44.922+0000 7fc40b4f4640 -1 reset not still connected to 0x5640bd0f1520 2024-04-23T21:50:44.924 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:50:44.922+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c14cbee0 2024-04-23T21:50:44.924 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:50:44.922+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c33ccc30 2024-04-23T21:50:44.924 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:50:44.922+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c348a680 2024-04-23T21:50:44.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:44.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:44.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:44.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_3 2024-04-23T21:50:44.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_3 snap 2024-04-23T21:50:44.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:44.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:44.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:50:44.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:44.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_3@snap 2024-04-23T21:50:45.481 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:45.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_3 2024-04-23T21:50:45.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:45.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:45.504 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:50:45.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:45.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:45.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_3 2024-04-23T21:50:45.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:45.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:45.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:50:45.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_3 2024-04-23T21:50:45.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_3 2024-04-23T21:50:45.626 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:50:45.626+0000 7f83e8352640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 3 2024-04-23T21:50:45.626 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:50:45.627+0000 7f83e9354640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 1 2024-04-23T21:50:46.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:46.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:46.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:46.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_4 2024-04-23T21:50:46.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_4 snap 2024-04-23T21:50:46.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:46.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:46.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:50:46.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:46.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_4@snap 2024-04-23T21:50:46.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:46.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_4 2024-04-23T21:50:46.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:46.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:46.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:50:46.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:46.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:46.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_4 2024-04-23T21:50:46.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:46.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:46.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:50:46.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_4 2024-04-23T21:50:46.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_4 2024-04-23T21:50:46.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:46.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:46.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:46.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_5 2024-04-23T21:50:46.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_5 snap 2024-04-23T21:50:46.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:46.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:46.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:50:46.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:46.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_5@snap 2024-04-23T21:50:47.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:47.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_5 2024-04-23T21:50:47.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:47.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:47.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:50:47.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:47.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:47.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_5 2024-04-23T21:50:47.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:47.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:47.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:50:47.496 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_5 2024-04-23T21:50:47.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_5 2024-04-23T21:50:47.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:50:47.599+0000 7f98854ae640 -1 rbd::mirror::ImageMap: 0x555f65bd9440 handle_peer_ack_remove: failed to remove global_image_id=f0ae1729-5015-402e-a870-d82f77ebb27b 2024-04-23T21:50:47.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:47.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:47.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:47.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_6 2024-04-23T21:50:47.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_6 snap 2024-04-23T21:50:47.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:47.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:47.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:50:47.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:47.756 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_6@snap 2024-04-23T21:50:48.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:48.493 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_6 2024-04-23T21:50:48.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:48.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:48.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:50:48.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:48.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:48.494 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_6 2024-04-23T21:50:48.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:48.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:48.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:50:48.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_6 2024-04-23T21:50:48.741 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_6 2024-04-23T21:50:48.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:50:48.805+0000 7f98854ae640 -1 rbd::mirror::ImageMap: 0x555f65bd9440 handle_peer_ack_remove: failed to remove global_image_id=943381b0-043f-49a7-8503-0581f0da0469 2024-04-23T21:50:48.936 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:48.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:48.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:48.939 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_7 2024-04-23T21:50:48.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_7 snap 2024-04-23T21:50:48.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:48.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:48.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:50:48.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:48.940 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_7@snap 2024-04-23T21:50:49.409 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2024-04-23T21:50:49.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2024-04-23T21:50:49.409 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2024-04-23T21:50:49.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2024-04-23T21:50:49.412 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2024-04-23T21:50:49.478 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:49.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_7 2024-04-23T21:50:49.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:49.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:49.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:50:49.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:49.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:49.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_7 2024-04-23T21:50:49.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:49.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:49.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:50:49.508 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_7 2024-04-23T21:50:49.562 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_7 2024-04-23T21:50:49.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:49.771 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_8 2024-04-23T21:50:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_8 snap 2024-04-23T21:50:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:50:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_8@snap 2024-04-23T21:50:50.489 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:50.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_8 2024-04-23T21:50:50.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:50.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:50.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:50:50.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:50.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:50.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_8 2024-04-23T21:50:50.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:50.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:50.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:50:50.507 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_8 2024-04-23T21:50:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_8 2024-04-23T21:50:50.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:50:50.611+0000 7f98854ae640 -1 rbd::mirror::ImageMap: 0x555f65bd9440 handle_peer_ack_remove: failed to remove global_image_id=91a8999d-21e8-48af-bea2-1a1cefa8f501 2024-04-23T21:50:50.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:50.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:50.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:50.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_9 2024-04-23T21:50:50.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_9 snap 2024-04-23T21:50:50.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:50.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:50.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:50:50.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:50.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_9@snap 2024-04-23T21:50:51.480 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:51.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_9 2024-04-23T21:50:51.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:51.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:51.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:50:51.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:51.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:51.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_9 2024-04-23T21:50:51.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:51.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:51.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:50:51.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_9 2024-04-23T21:50:51.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_9 2024-04-23T21:50:51.760 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:51.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:51.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:51.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_10 2024-04-23T21:50:51.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_10 snap 2024-04-23T21:50:51.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:51.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:51.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:50:51.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:51.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_10@snap 2024-04-23T21:50:52.393 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:52.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_10 2024-04-23T21:50:52.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:52.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:52.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:50:52.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:52.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:52.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_10 2024-04-23T21:50:52.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:52.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:52.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:50:52.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_10 2024-04-23T21:50:52.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_10 2024-04-23T21:50:52.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:50:52.519+0000 7f9885caf640 -1 rbd::mirror::ImageMap: 0x555f65bd9440 handle_peer_ack_remove: failed to remove global_image_id=a5003cc7-3a00-4b0f-b660-c9391be7ce73 2024-04-23T21:50:52.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:52.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:52.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:52.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_11 2024-04-23T21:50:52.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_11 snap 2024-04-23T21:50:52.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:52.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:52.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:50:52.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:52.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_11@snap 2024-04-23T21:50:53.495 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_11 2024-04-23T21:50:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:50:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:53.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:53.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_11 2024-04-23T21:50:53.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:53.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:53.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:50:53.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_11 2024-04-23T21:50:53.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_11 2024-04-23T21:50:53.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:53.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:53.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:53.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_12 2024-04-23T21:50:53.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_12 snap 2024-04-23T21:50:53.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:53.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:53.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:50:53.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:53.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_12@snap 2024-04-23T21:50:54.497 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:54.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_12 2024-04-23T21:50:54.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:54.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:54.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:50:54.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:54.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:54.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_12 2024-04-23T21:50:54.516 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:54.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:54.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:50:54.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_12 2024-04-23T21:50:54.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_12 2024-04-23T21:50:54.635 INFO:tasks.ceph.cluster1.osd.1.smithi148.stderr:2024-04-23T21:50:54.635+0000 7f32b7199640 -1 reset not still connected to 0x561826fb1930 2024-04-23T21:50:54.750 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:54.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:54.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:54.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_13 2024-04-23T21:50:54.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_13 snap 2024-04-23T21:50:54.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:54.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:54.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:50:54.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:54.755 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_13@snap 2024-04-23T21:50:55.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:55.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_13 2024-04-23T21:50:55.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:55.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:55.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:50:55.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:55.524 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:55.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_13 2024-04-23T21:50:55.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:55.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:55.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:50:55.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_13 2024-04-23T21:50:55.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_13 2024-04-23T21:50:55.984 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:55.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:55.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:55.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_14 2024-04-23T21:50:55.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_14 snap 2024-04-23T21:50:55.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:55.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:55.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:50:55.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:55.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_14@snap 2024-04-23T21:50:56.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:56.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_14 2024-04-23T21:50:56.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:56.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:56.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:50:56.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:56.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:56.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_14 2024-04-23T21:50:56.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:56.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:56.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:50:56.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_14 2024-04-23T21:50:56.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_14 2024-04-23T21:50:56.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:50:56.635+0000 7f9885caf640 -1 rbd::mirror::ImageMap: 0x555f65bd9440 handle_peer_ack_remove: failed to remove global_image_id=da53d3c5-7d2f-44c2-99f8-4c19141f4790 2024-04-23T21:50:56.674 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:50:56.673+0000 7f83ed35c640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 10 2024-04-23T21:50:56.676 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:50:56.675+0000 7f83e7350640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 17 2024-04-23T21:50:56.676 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:50:56.675+0000 7f83ecb5b640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 16 2024-04-23T21:50:56.884 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:56.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:56.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:56.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_15 2024-04-23T21:50:56.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_15 snap 2024-04-23T21:50:56.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:56.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:56.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:50:56.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:56.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_15@snap 2024-04-23T21:50:57.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:57.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_15 2024-04-23T21:50:57.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:57.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:57.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:50:57.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:57.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:57.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_15 2024-04-23T21:50:57.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:57.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:57.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:50:57.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_15 2024-04-23T21:50:57.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_15 2024-04-23T21:50:57.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:50:57.527+0000 7f98854ae640 -1 rbd::mirror::ImageMap: 0x555f65bd9440 handle_peer_ack_remove: failed to remove global_image_id=cd904466-1c61-4f0b-869a-10525a3dcdf0 2024-04-23T21:50:57.672 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:57.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:57.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:57.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_16 2024-04-23T21:50:57.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_16 snap 2024-04-23T21:50:57.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:57.676 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:57.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:50:57.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:57.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_16@snap 2024-04-23T21:50:57.735 INFO:tasks.ceph.cluster2.osd.1.smithi150.stderr:2024-04-23T21:50:57.734+0000 7f5865e11640 -1 bdev(0x559fd0290e00 /var/lib/ceph/osd/cluster2-1/block) aio_submit retries 1 2024-04-23T21:50:58.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_16 2024-04-23T21:50:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:58.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:50:58.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:58.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:58.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_16 2024-04-23T21:50:58.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:58.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:58.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:50:58.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_16 2024-04-23T21:50:58.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_16 2024-04-23T21:50:58.798 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:58.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:58.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:58.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_17 2024-04-23T21:50:58.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_17 snap 2024-04-23T21:50:58.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:58.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:58.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:50:58.802 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:58.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_17@snap 2024-04-23T21:50:59.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:50:59.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_17 2024-04-23T21:50:59.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:59.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:59.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:50:59.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:50:59.536 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:50:59.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_17 2024-04-23T21:50:59.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:59.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:59.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:50:59.537 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_17 2024-04-23T21:50:59.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_17 2024-04-23T21:50:59.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:50:59.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:50:59.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:50:59.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_18 2024-04-23T21:50:59.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_18 snap 2024-04-23T21:50:59.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:50:59.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:50:59.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:50:59.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:50:59.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_18@snap 2024-04-23T21:51:00.505 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:00.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_18 2024-04-23T21:51:00.519 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:00.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:00.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:51:00.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:00.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:00.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_18 2024-04-23T21:51:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:51:00.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_18 2024-04-23T21:51:00.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_18 2024-04-23T21:51:00.792 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:00.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:00.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:00.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_19 2024-04-23T21:51:00.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_19 snap 2024-04-23T21:51:00.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:00.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:00.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:51:00.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:00.796 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_19@snap 2024-04-23T21:51:01.520 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:01.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_19 2024-04-23T21:51:01.538 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:01.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:01.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:51:01.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:01.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:01.539 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_19 2024-04-23T21:51:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:51:01.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_19 2024-04-23T21:51:01.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_19 2024-04-23T21:51:01.772 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:01.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:01.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:01.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_20 2024-04-23T21:51:01.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_20 snap 2024-04-23T21:51:01.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:01.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:01.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:51:01.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:01.776 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_20@snap 2024-04-23T21:51:02.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:02.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_20 2024-04-23T21:51:02.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:02.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:02.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:51:02.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:02.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:02.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_20 2024-04-23T21:51:02.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:02.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:02.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:51:02.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_20 2024-04-23T21:51:02.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_20 2024-04-23T21:51:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:02.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:02.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:02.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_21 2024-04-23T21:51:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_21 snap 2024-04-23T21:51:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:51:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_21@snap 2024-04-23T21:51:02.753 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:51:02.752+0000 7f83e8b53640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 2 2024-04-23T21:51:02.754 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:51:02.753+0000 7f83ea356640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 6 2024-04-23T21:51:03.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:03.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_21 2024-04-23T21:51:03.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:03.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:03.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:51:03.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:03.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:03.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_21 2024-04-23T21:51:03.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:03.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:03.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:51:03.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_21 2024-04-23T21:51:03.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_21 2024-04-23T21:51:03.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_22 2024-04-23T21:51:03.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_22 snap 2024-04-23T21:51:03.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:03.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:03.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:51:03.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:03.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_22@snap 2024-04-23T21:51:04.530 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_22 2024-04-23T21:51:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:51:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:04.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:04.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_22 2024-04-23T21:51:04.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:04.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:04.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:51:04.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_22 2024-04-23T21:51:04.588 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_22 2024-04-23T21:51:04.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:04.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:04.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:04.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_23 2024-04-23T21:51:04.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_23 snap 2024-04-23T21:51:04.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:04.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:04.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:51:04.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:04.809 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_23@snap 2024-04-23T21:51:05.526 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:05.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_23 2024-04-23T21:51:05.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:05.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:05.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:51:05.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:05.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:05.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_23 2024-04-23T21:51:05.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:05.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:05.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:51:05.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_23 2024-04-23T21:51:05.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_23 2024-04-23T21:51:05.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:05.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:05.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:05.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_24 2024-04-23T21:51:05.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_24 snap 2024-04-23T21:51:05.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:05.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:05.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:51:05.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:05.795 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_24@snap 2024-04-23T21:51:06.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:06.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_24 2024-04-23T21:51:06.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:06.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:06.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:51:06.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:06.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:06.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_24 2024-04-23T21:51:06.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:06.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:06.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:51:06.555 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_24 2024-04-23T21:51:06.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_24 2024-04-23T21:51:06.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:06.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:06.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:06.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_25 2024-04-23T21:51:06.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_25 snap 2024-04-23T21:51:06.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:06.787 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:06.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:51:06.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:06.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_25@snap 2024-04-23T21:51:07.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:07.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_25 2024-04-23T21:51:07.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:07.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:07.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:51:07.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:07.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:07.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_25 2024-04-23T21:51:07.414 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:07.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:07.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:51:07.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_25 2024-04-23T21:51:07.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_25 2024-04-23T21:51:07.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_26 2024-04-23T21:51:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_26 snap 2024-04-23T21:51:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:51:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:07.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_26@snap 2024-04-23T21:51:08.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:08.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_26 2024-04-23T21:51:08.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:08.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:08.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:51:08.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:08.567 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:08.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_26 2024-04-23T21:51:08.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:08.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:08.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:51:08.568 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_26 2024-04-23T21:51:08.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_26 2024-04-23T21:51:08.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:51:08.675+0000 7f9885caf640 -1 rbd::mirror::ImageMap: 0x555f65bd9440 handle_peer_ack_remove: failed to remove global_image_id=ef4cdba8-fa7e-4c68-a518-af82a968b35a 2024-04-23T21:51:08.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:08.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:08.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:08.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_27 2024-04-23T21:51:08.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_27 snap 2024-04-23T21:51:08.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:08.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:08.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:51:08.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:08.808 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_27@snap 2024-04-23T21:51:09.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:09.552 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_27 2024-04-23T21:51:09.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:09.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:09.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:51:09.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:09.553 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:09.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_27 2024-04-23T21:51:09.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:09.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:09.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:51:09.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_27 2024-04-23T21:51:09.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_27 2024-04-23T21:51:09.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:51:09.681+0000 7f98854ae640 -1 rbd::mirror::ImageMap: 0x555f65bd9440 handle_peer_ack_remove: failed to remove global_image_id=32be9fcc-7049-47d4-813d-6edb9a876f43 2024-04-23T21:51:09.807 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:09.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:09.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:09.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_28 2024-04-23T21:51:09.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_28 snap 2024-04-23T21:51:09.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:09.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:09.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:51:09.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:09.811 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_28@snap 2024-04-23T21:51:10.554 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:10.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_28 2024-04-23T21:51:10.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:10.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:10.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:51:10.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:10.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:10.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_28 2024-04-23T21:51:10.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:10.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:10.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:51:10.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_28 2024-04-23T21:51:10.620 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_28 2024-04-23T21:51:10.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:10.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:10.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:10.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_29 2024-04-23T21:51:10.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_29 snap 2024-04-23T21:51:10.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:10.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:10.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:51:10.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:10.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_29@snap 2024-04-23T21:51:11.566 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:11.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_29 2024-04-23T21:51:11.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:11.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:11.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:51:11.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:11.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:11.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_29 2024-04-23T21:51:11.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:11.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:11.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:51:11.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_29 2024-04-23T21:51:11.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_29 2024-04-23T21:51:11.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:11.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:11.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:11.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_30 2024-04-23T21:51:11.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_30 snap 2024-04-23T21:51:11.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:11.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:11.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:51:11.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:11.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_30@snap 2024-04-23T21:51:12.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:12.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_30 2024-04-23T21:51:12.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:12.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:12.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:51:12.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:12.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:12.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_30 2024-04-23T21:51:12.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:12.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:12.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:51:12.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_30 2024-04-23T21:51:12.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_30 2024-04-23T21:51:12.652 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_31 2024-04-23T21:51:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_31 snap 2024-04-23T21:51:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:51:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:12.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_31@snap 2024-04-23T21:51:13.564 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:13.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_31 2024-04-23T21:51:13.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:13.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:13.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:51:13.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:13.591 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:13.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_31 2024-04-23T21:51:13.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:13.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:13.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:51:13.592 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_31 2024-04-23T21:51:13.841 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_31 2024-04-23T21:51:13.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:51:13.907+0000 7f9885caf640 -1 rbd::mirror::ImageMap: 0x555f65bd9440 handle_peer_ack_remove: failed to remove global_image_id=fbd02d69-b6c8-4f22-87f2-b319996d2191 2024-04-23T21:51:14.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:14.040 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:14.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_32 2024-04-23T21:51:14.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_32 snap 2024-04-23T21:51:14.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:14.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:14.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:51:14.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:14.041 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_32@snap 2024-04-23T21:51:14.561 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:14.574 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_32 2024-04-23T21:51:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:51:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:14.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:14.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_32 2024-04-23T21:51:14.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:14.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:14.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:51:14.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_32 2024-04-23T21:51:14.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_32 2024-04-23T21:51:14.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:14.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:14.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:14.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_33 2024-04-23T21:51:14.839 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_33 snap 2024-04-23T21:51:14.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:14.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:14.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:51:14.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:14.840 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_33@snap 2024-04-23T21:51:15.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:15.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_33 2024-04-23T21:51:15.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:15.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:15.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:51:15.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:15.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:15.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_33 2024-04-23T21:51:15.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:15.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:15.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:51:15.599 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_33 2024-04-23T21:51:15.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_33 2024-04-23T21:51:15.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_34 2024-04-23T21:51:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_34 snap 2024-04-23T21:51:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:51:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_34@snap 2024-04-23T21:51:16.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:16.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_34 2024-04-23T21:51:16.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:16.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:16.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:51:16.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:16.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_34 2024-04-23T21:51:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:51:16.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_34 2024-04-23T21:51:16.726 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_34 2024-04-23T21:51:16.888 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:51:16.889+0000 7f83fbfb5640 -1 reset not still connected to 0x562a5ceb7110 2024-04-23T21:51:16.944 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:16.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:16.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:16.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_35 2024-04-23T21:51:16.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_35 snap 2024-04-23T21:51:16.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:16.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:16.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:51:16.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:16.949 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_35@snap 2024-04-23T21:51:17.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:17.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_35 2024-04-23T21:51:17.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:17.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:17.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:51:17.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:17.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:17.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_35 2024-04-23T21:51:17.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:17.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:17.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:51:17.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_35 2024-04-23T21:51:17.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_35 2024-04-23T21:51:17.650 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:17.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:17.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:17.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_36 2024-04-23T21:51:17.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_36 snap 2024-04-23T21:51:17.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:17.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:17.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:51:17.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:17.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_36@snap 2024-04-23T21:51:18.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:18.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_36 2024-04-23T21:51:18.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:18.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:18.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:51:18.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:18.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:18.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_36 2024-04-23T21:51:18.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:18.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:18.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:51:18.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_36 2024-04-23T21:51:18.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_36 2024-04-23T21:51:18.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:18.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:18.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:18.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_37 2024-04-23T21:51:18.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_37 snap 2024-04-23T21:51:18.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:18.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:18.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:51:18.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:18.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_37@snap 2024-04-23T21:51:19.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:19.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_37 2024-04-23T21:51:19.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:19.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:19.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:51:19.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:19.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:19.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_37 2024-04-23T21:51:19.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:19.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:19.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:51:19.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_37 2024-04-23T21:51:19.657 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_37 2024-04-23T21:51:19.860 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_38 2024-04-23T21:51:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_38 snap 2024-04-23T21:51:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:51:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_38@snap 2024-04-23T21:51:20.587 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_38 2024-04-23T21:51:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:51:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:20.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:20.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_38 2024-04-23T21:51:20.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:20.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:20.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:51:20.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_38 2024-04-23T21:51:20.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_38 2024-04-23T21:51:20.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:20.914 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:20.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:20.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_39 2024-04-23T21:51:20.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_39 snap 2024-04-23T21:51:20.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:20.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:20.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:51:20.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:20.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_39@snap 2024-04-23T21:51:21.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:21.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_39 2024-04-23T21:51:21.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:21.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:21.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:51:21.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:21.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:21.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_39 2024-04-23T21:51:21.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:21.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:21.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:51:21.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_39 2024-04-23T21:51:21.677 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_39 2024-04-23T21:51:22.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:22.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:22.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:22.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_40 2024-04-23T21:51:22.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_40 snap 2024-04-23T21:51:22.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:22.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:22.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:51:22.091 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:22.092 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_40@snap 2024-04-23T21:51:22.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:22.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_40 2024-04-23T21:51:22.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:22.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:22.416 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:51:22.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:22.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:22.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_40 2024-04-23T21:51:22.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:22.417 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:22.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:51:22.418 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_40 2024-04-23T21:51:22.462 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_40 2024-04-23T21:51:22.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:22.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:22.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:22.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_41 2024-04-23T21:51:22.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_41 snap 2024-04-23T21:51:22.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:22.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:22.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:51:22.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:22.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_41@snap 2024-04-23T21:51:23.619 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:23.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_41 2024-04-23T21:51:23.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:23.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:23.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:51:23.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:23.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:23.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_41 2024-04-23T21:51:23.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:23.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:23.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:51:23.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_41 2024-04-23T21:51:23.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_41 2024-04-23T21:51:23.885 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:23.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:23.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:23.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_42 2024-04-23T21:51:23.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_42 snap 2024-04-23T21:51:23.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:23.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:23.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:51:23.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:23.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_42@snap 2024-04-23T21:51:24.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:24.654 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_42 2024-04-23T21:51:24.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:24.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:24.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:51:24.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:24.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:24.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_42 2024-04-23T21:51:24.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:24.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:24.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:51:24.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_42 2024-04-23T21:51:24.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_42 2024-04-23T21:51:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:24.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:24.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:24.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_43 2024-04-23T21:51:24.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_43 snap 2024-04-23T21:51:24.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:24.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:24.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:51:24.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:24.911 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_43@snap 2024-04-23T21:51:25.644 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:25.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_43 2024-04-23T21:51:25.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:25.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:25.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:51:25.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:25.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:25.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_43 2024-04-23T21:51:25.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:25.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:25.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:51:25.664 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_43 2024-04-23T21:51:25.709 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_43 2024-04-23T21:51:25.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_44 2024-04-23T21:51:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_44 snap 2024-04-23T21:51:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:51:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:25.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_44@snap 2024-04-23T21:51:26.638 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:26.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_44 2024-04-23T21:51:26.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:26.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:26.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:51:26.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:26.661 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:26.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_44 2024-04-23T21:51:26.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:26.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:26.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:51:26.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_44 2024-04-23T21:51:26.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_44 2024-04-23T21:51:26.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:26.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:26.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:26.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_45 2024-04-23T21:51:26.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_45 snap 2024-04-23T21:51:26.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:26.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:26.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:51:26.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:26.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_45@snap 2024-04-23T21:51:27.409 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:27.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_45 2024-04-23T21:51:27.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:27.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:27.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:51:27.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:27.426 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:27.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_45 2024-04-23T21:51:27.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:27.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:27.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:51:27.427 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_45 2024-04-23T21:51:27.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_45 2024-04-23T21:51:27.655 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:27.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:27.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:27.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_46 2024-04-23T21:51:27.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_46 snap 2024-04-23T21:51:27.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:27.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:27.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:51:27.659 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:27.660 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_46@snap 2024-04-23T21:51:27.847 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:51:27.847+0000 7fc40b4f4640 -1 reset not still connected to 0x5640b54881a0 2024-04-23T21:51:27.847 INFO:tasks.ceph.cluster1.osd.0.smithi148.stderr:2024-04-23T21:51:27.847+0000 7fc40b4f4640 -1 reset not still connected to 0x5640c14cbee0 2024-04-23T21:51:28.641 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:28.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_46 2024-04-23T21:51:28.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:28.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:28.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:51:28.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:28.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:28.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_46 2024-04-23T21:51:28.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:28.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:28.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:51:28.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_46 2024-04-23T21:51:28.706 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_46 2024-04-23T21:51:28.904 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:28.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:28.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:28.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_47 2024-04-23T21:51:28.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_47 snap 2024-04-23T21:51:28.908 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:28.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:28.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:51:28.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:28.909 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_47@snap 2024-04-23T21:51:29.665 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:29.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_47 2024-04-23T21:51:29.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:29.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:29.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:51:29.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:29.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:29.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_47 2024-04-23T21:51:29.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:29.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:29.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:51:29.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_47 2024-04-23T21:51:29.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_47 2024-04-23T21:51:29.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:29.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:29.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:29.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_48 2024-04-23T21:51:29.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_48 snap 2024-04-23T21:51:29.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:29.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:29.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:51:29.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:29.935 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_48@snap 2024-04-23T21:51:30.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_48 2024-04-23T21:51:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:51:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:30.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:30.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_48 2024-04-23T21:51:30.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:30.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:30.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:51:30.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_48 2024-04-23T21:51:30.725 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_48 2024-04-23T21:51:30.913 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:30.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:30.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:30.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_49 2024-04-23T21:51:30.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_49 snap 2024-04-23T21:51:30.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:30.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:30.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:51:30.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:30.917 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_49@snap 2024-04-23T21:51:31.662 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:31.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_49 2024-04-23T21:51:31.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:31.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:31.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:51:31.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:31.686 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:31.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_49 2024-04-23T21:51:31.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:31.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:31.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:51:31.687 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_49 2024-04-23T21:51:31.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_49 2024-04-23T21:51:31.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:31.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:31.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:31.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_50 2024-04-23T21:51:31.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_snapshot cluster2 mirror image_50 snap 2024-04-23T21:51:31.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:31.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:31.933 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:51:31.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local snap=snap 2024-04-23T21:51:31.934 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap rm mirror/image_50@snap 2024-04-23T21:51:32.403 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing snap: 100% complete...done. 2024-04-23T21:51:32.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_50 2024-04-23T21:51:32.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:32.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:32.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:51:32.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:51:32.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:51:32.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_50 2024-04-23T21:51:32.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:32.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:32.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:51:32.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_50 2024-04-23T21:51:32.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_50 2024-04-23T21:51:32.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2024-04-23T21:51:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ testlog 'TEST: image deletions should propagate' 2024-04-23T21:51:32.698 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ tee -a /tmp/tmp.8Tyc9aSFBO/rbd-mirror.test.log 2024-04-23T21:51:32.699 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ date '+%F %T' 2024-04-23T21:51:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo 2024-04-23 21:51:32 TEST: image deletions should propagate 2024-04-23T21:51:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:2024-04-23 21:51:32 TEST: image deletions should propagate 2024-04-23T21:51:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_pool_images cluster1 mirror 0 2024-04-23T21:51:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_count=0 2024-04-23T21:51:32.700 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:32.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local count 2024-04-23T21:51:32.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local last_count=0 2024-04-23T21:51:32.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ true 2024-04-23T21:51:32.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 1 40 2024-04-23T21:51:32.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in `seq 1 40` 2024-04-23T21:51:32.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 1 -ne 1 2024-04-23T21:51:32.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 -p mirror mirror pool status 2024-04-23T21:51:32.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep 'images: ' 2024-04-23T21:51:32.702 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut '-d ' -f 2 2024-04-23T21:51:32.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: failed to query services: (13) Permission denied 2024-04-23T21:51:32.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ count=0 2024-04-23T21:51:32.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 0 = 0 2024-04-23T21:51:32.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:32.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_pool_healthy cluster1 mirror 0 2024-04-23T21:51:32.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:32.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:32.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:32.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state 2024-04-23T21:51:32.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 1 40 2024-04-23T21:51:32.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in `seq 1 40` 2024-04-23T21:51:32.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 1 -ne 1 2024-04-23T21:51:32.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster cluster1 -p mirror mirror pool status 2024-04-23T21:51:32.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep 'image health:' 2024-04-23T21:51:32.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cut '-d ' -f 3 2024-04-23T21:51:32.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: failed to query services: (13) Permission denied 2024-04-23T21:51:32.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ state=OK 2024-04-23T21:51:32.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test OK = ERROR 2024-04-23T21:51:32.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test OK = OK 2024-04-23T21:51:32.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:32.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 1 50 2024-04-23T21:51:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_1 2024-04-23T21:51:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_1 deleted 2024-04-23T21:51:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:51:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_1 2024-04-23T21:51:32.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:32.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:32.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_1 2024-04-23T21:51:32.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_1 2024-04-23T21:51:32.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:32.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_1: (2) No such file or directory 2024-04-23T21:51:32.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:32.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:32.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:32.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_1 deleted '' 2024-04-23T21:51:32.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:32.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:32.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:51:32.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:32.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:32.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:32.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:32.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_1 2024-04-23T21:51:32.924 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:32.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:32.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_1 2024-04-23T21:51:32.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_1 2024-04-23T21:51:32.925 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:32.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_1: (2) No such file or directory 2024-04-23T21:51:32.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:32.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:32.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:32.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:32.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:32.959 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_2 2024-04-23T21:51:32.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_2 deleted 2024-04-23T21:51:32.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:32.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:32.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:51:32.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:32.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:32.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:32.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:32.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_2 2024-04-23T21:51:32.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:32.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:32.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:51:32.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_2 2024-04-23T21:51:32.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:32.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_2: (2) No such file or directory 2024-04-23T21:51:32.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:32.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:32.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_2 deleted '' 2024-04-23T21:51:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:51:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:33.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:33.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:33.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_2 2024-04-23T21:51:33.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:33.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:33.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:51:33.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_2 2024-04-23T21:51:33.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:33.126 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_2: (2) No such file or directory 2024-04-23T21:51:33.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_3 2024-04-23T21:51:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_3 deleted 2024-04-23T21:51:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:51:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:33.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:33.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_3 2024-04-23T21:51:33.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:33.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:33.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:51:33.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_3 2024-04-23T21:51:33.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:33.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_3: (2) No such file or directory 2024-04-23T21:51:33.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:33.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:33.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:33.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_3 deleted '' 2024-04-23T21:51:33.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:33.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:33.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:51:33.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:33.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:33.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:33.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:33.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_3 2024-04-23T21:51:33.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:33.267 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:33.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:51:33.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_3 2024-04-23T21:51:33.268 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:33.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_3: (2) No such file or directory 2024-04-23T21:51:33.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:33.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:33.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:33.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:33.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:33.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_4 2024-04-23T21:51:33.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_4 deleted 2024-04-23T21:51:33.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:33.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:33.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:51:33.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:33.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:33.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:33.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:33.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_4 2024-04-23T21:51:33.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:33.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:33.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:51:33.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_4 2024-04-23T21:51:33.304 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:33.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_4: (2) No such file or directory 2024-04-23T21:51:33.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:33.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:33.337 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:33.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_4 deleted '' 2024-04-23T21:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_4 2024-04-23T21:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_4 2024-04-23T21:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:33.470 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_4: (2) No such file or directory 2024-04-23T21:51:33.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:33.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:33.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:33.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:33.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:33.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_5 2024-04-23T21:51:33.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_5 deleted 2024-04-23T21:51:33.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:33.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:33.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:51:33.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_5 2024-04-23T21:51:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:51:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_5 2024-04-23T21:51:33.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:33.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_5: (2) No such file or directory 2024-04-23T21:51:33.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:33.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:33.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:33.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_5 deleted '' 2024-04-23T21:51:33.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:33.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:33.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:51:33.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:33.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:33.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:33.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:33.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_5 2024-04-23T21:51:33.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:33.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:33.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:51:33.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_5 2024-04-23T21:51:33.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:33.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_5: (2) No such file or directory 2024-04-23T21:51:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:33.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_6 2024-04-23T21:51:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_6 deleted 2024-04-23T21:51:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:51:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_6 2024-04-23T21:51:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_6 2024-04-23T21:51:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_6 2024-04-23T21:51:33.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:33.678 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_6: (2) No such file or directory 2024-04-23T21:51:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:33.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:33.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_6 deleted '' 2024-04-23T21:51:33.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:33.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:33.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:51:33.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:33.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:33.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:33.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:33.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_6 2024-04-23T21:51:33.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:33.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:33.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_6 2024-04-23T21:51:33.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_6 2024-04-23T21:51:33.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:33.814 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_6: (2) No such file or directory 2024-04-23T21:51:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_7 2024-04-23T21:51:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_7 deleted 2024-04-23T21:51:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:51:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:33.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:33.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:33.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:33.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_7 2024-04-23T21:51:33.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:33.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:33.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:51:33.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_7 2024-04-23T21:51:33.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:33.851 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_7: (2) No such file or directory 2024-04-23T21:51:33.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:33.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:33.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:33.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_7 deleted '' 2024-04-23T21:51:33.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:33.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:33.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:51:33.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:33.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:33.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:33.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:33.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_7 2024-04-23T21:51:33.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:33.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:33.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:51:33.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_7 2024-04-23T21:51:33.957 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:33.988 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_7: (2) No such file or directory 2024-04-23T21:51:33.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:33.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:33.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:33.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:33.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:33.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_8 2024-04-23T21:51:33.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_8 deleted 2024-04-23T21:51:33.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:33.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:33.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:51:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_8 2024-04-23T21:51:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:51:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_8 2024-04-23T21:51:33.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:34.024 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_8: (2) No such file or directory 2024-04-23T21:51:34.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:34.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:34.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:34.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_8 deleted '' 2024-04-23T21:51:34.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:34.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:34.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:51:34.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:34.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:34.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:34.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:34.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_8 2024-04-23T21:51:34.129 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:34.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:34.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:51:34.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_8 2024-04-23T21:51:34.130 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:34.161 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_8: (2) No such file or directory 2024-04-23T21:51:34.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:34.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:34.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_9 2024-04-23T21:51:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_9 deleted 2024-04-23T21:51:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:51:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_9 2024-04-23T21:51:34.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:51:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_9 2024-04-23T21:51:34.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:34.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_9: (2) No such file or directory 2024-04-23T21:51:34.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:34.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:34.201 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:34.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_9 deleted '' 2024-04-23T21:51:34.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:34.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:34.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:51:34.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:34.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:34.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:34.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:34.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_9 2024-04-23T21:51:34.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:34.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:34.306 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:51:34.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_9 2024-04-23T21:51:34.307 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:34.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_9: (2) No such file or directory 2024-04-23T21:51:34.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:34.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:34.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:34.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:34.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:34.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_10 2024-04-23T21:51:34.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_10 deleted 2024-04-23T21:51:34.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:34.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:34.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:51:34.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:34.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:34.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:34.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:34.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_10 2024-04-23T21:51:34.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:34.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:34.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:51:34.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_10 2024-04-23T21:51:34.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:34.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_10: (2) No such file or directory 2024-04-23T21:51:34.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:34.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:34.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:34.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_10 deleted '' 2024-04-23T21:51:34.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:34.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:34.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:51:34.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:34.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:34.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:34.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:34.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_10 2024-04-23T21:51:34.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:34.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:34.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:51:34.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_10 2024-04-23T21:51:34.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:34.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_10: (2) No such file or directory 2024-04-23T21:51:34.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:34.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:34.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:34.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:34.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_11 2024-04-23T21:51:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_11 deleted 2024-04-23T21:51:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:51:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_11 2024-04-23T21:51:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:51:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_11 2024-04-23T21:51:34.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:34.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_11: (2) No such file or directory 2024-04-23T21:51:34.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:34.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:34.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:34.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_11 deleted '' 2024-04-23T21:51:34.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:34.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:34.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_11 2024-04-23T21:51:34.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:34.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:34.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:34.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:34.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_11 2024-04-23T21:51:34.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:34.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:34.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_11 2024-04-23T21:51:34.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_11 2024-04-23T21:51:34.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:34.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_11: (2) No such file or directory 2024-04-23T21:51:34.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:34.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:34.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:34.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:34.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:34.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_12 2024-04-23T21:51:34.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_12 deleted 2024-04-23T21:51:34.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:34.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:34.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:51:34.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:34.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:34.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:34.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:34.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_12 2024-04-23T21:51:34.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:34.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:34.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:51:34.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_12 2024-04-23T21:51:34.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:34.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_12: (2) No such file or directory 2024-04-23T21:51:34.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:34.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:34.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:34.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_12 deleted '' 2024-04-23T21:51:34.818 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_12 2024-04-23T21:51:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_12 2024-04-23T21:51:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_12 2024-04-23T21:51:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_12 2024-04-23T21:51:34.819 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:34.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_12: (2) No such file or directory 2024-04-23T21:51:34.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:34.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:34.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:34.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:34.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:34.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_13 2024-04-23T21:51:34.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_13 deleted 2024-04-23T21:51:34.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:34.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:34.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:51:34.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:34.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:34.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:34.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:34.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_13 2024-04-23T21:51:34.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:34.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:34.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:51:34.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_13 2024-04-23T21:51:34.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:34.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_13: (2) No such file or directory 2024-04-23T21:51:34.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:34.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:34.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:34.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_13 deleted '' 2024-04-23T21:51:34.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:34.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_13 2024-04-23T21:51:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_13 2024-04-23T21:51:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_13 2024-04-23T21:51:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_13 2024-04-23T21:51:34.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:35.025 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_13: (2) No such file or directory 2024-04-23T21:51:35.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:35.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_14 2024-04-23T21:51:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_14 deleted 2024-04-23T21:51:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:51:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:35.030 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_14 2024-04-23T21:51:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:51:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_14 2024-04-23T21:51:35.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:35.060 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_14: (2) No such file or directory 2024-04-23T21:51:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:35.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:35.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_14 deleted '' 2024-04-23T21:51:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_14 2024-04-23T21:51:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_14 2024-04-23T21:51:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:35.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_14 2024-04-23T21:51:35.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_14 2024-04-23T21:51:35.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:35.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_14: (2) No such file or directory 2024-04-23T21:51:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_15 2024-04-23T21:51:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_15 deleted 2024-04-23T21:51:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:51:35.202 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:35.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:35.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:35.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:35.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_15 2024-04-23T21:51:35.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:35.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:35.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_15 2024-04-23T21:51:35.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_15 2024-04-23T21:51:35.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:35.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_15: (2) No such file or directory 2024-04-23T21:51:35.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:35.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:35.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_15 deleted '' 2024-04-23T21:51:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_15 2024-04-23T21:51:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_15 2024-04-23T21:51:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_15 2024-04-23T21:51:35.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_15 2024-04-23T21:51:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:35.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_15: (2) No such file or directory 2024-04-23T21:51:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_16 2024-04-23T21:51:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_16 deleted 2024-04-23T21:51:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:51:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:35.374 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:35.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:35.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:35.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_16 2024-04-23T21:51:35.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:35.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:35.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:51:35.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_16 2024-04-23T21:51:35.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:35.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_16: (2) No such file or directory 2024-04-23T21:51:35.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:35.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:35.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_16 deleted '' 2024-04-23T21:51:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_16 2024-04-23T21:51:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_16 2024-04-23T21:51:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:35.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_16 2024-04-23T21:51:35.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_16 2024-04-23T21:51:35.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:35.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_16: (2) No such file or directory 2024-04-23T21:51:35.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_17 2024-04-23T21:51:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_17 deleted 2024-04-23T21:51:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:51:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_17 2024-04-23T21:51:35.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:35.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:35.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:51:35.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_17 2024-04-23T21:51:35.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:35.575 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_17: (2) No such file or directory 2024-04-23T21:51:35.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:35.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:35.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:35.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_17 deleted '' 2024-04-23T21:51:35.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_17 2024-04-23T21:51:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_17 2024-04-23T21:51:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_17 2024-04-23T21:51:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_17 2024-04-23T21:51:35.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:35.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_17: (2) No such file or directory 2024-04-23T21:51:35.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:35.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:35.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:35.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:35.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:35.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_18 2024-04-23T21:51:35.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_18 deleted 2024-04-23T21:51:35.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:35.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:35.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:51:35.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:35.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:35.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:35.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:35.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_18 2024-04-23T21:51:35.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:35.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:35.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:51:35.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_18 2024-04-23T21:51:35.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:35.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_18: (2) No such file or directory 2024-04-23T21:51:35.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:35.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:35.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:35.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_18 deleted '' 2024-04-23T21:51:35.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:35.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:35.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_18 2024-04-23T21:51:35.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:35.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:35.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:35.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:35.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_18 2024-04-23T21:51:35.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:35.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:35.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_18 2024-04-23T21:51:35.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_18 2024-04-23T21:51:35.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:35.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_18: (2) No such file or directory 2024-04-23T21:51:35.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:35.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:35.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:35.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:35.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:35.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_19 2024-04-23T21:51:35.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_19 deleted 2024-04-23T21:51:35.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:35.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:35.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:51:35.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:35.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:35.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:35.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:35.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_19 2024-04-23T21:51:35.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:35.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:35.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:51:35.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_19 2024-04-23T21:51:35.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:35.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_19: (2) No such file or directory 2024-04-23T21:51:35.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:35.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:35.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:36.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_19 deleted '' 2024-04-23T21:51:36.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:36.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:36.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_19 2024-04-23T21:51:36.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:36.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:36.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:36.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:36.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_19 2024-04-23T21:51:36.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:36.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:36.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_19 2024-04-23T21:51:36.034 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_19 2024-04-23T21:51:36.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:36.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_19: (2) No such file or directory 2024-04-23T21:51:36.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:36.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:36.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:36.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:36.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:36.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_20 2024-04-23T21:51:36.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_20 deleted 2024-04-23T21:51:36.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:36.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:36.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:51:36.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:36.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:36.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:36.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:36.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_20 2024-04-23T21:51:36.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:36.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:36.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:51:36.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_20 2024-04-23T21:51:36.070 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:36.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_20: (2) No such file or directory 2024-04-23T21:51:36.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:36.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:36.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:36.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_20 deleted '' 2024-04-23T21:51:36.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:36.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:36.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_20 2024-04-23T21:51:36.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:36.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:36.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:36.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:36.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_20 2024-04-23T21:51:36.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:36.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:36.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_20 2024-04-23T21:51:36.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_20 2024-04-23T21:51:36.205 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:36.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_20: (2) No such file or directory 2024-04-23T21:51:36.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:36.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:36.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:36.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:36.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:36.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_21 2024-04-23T21:51:36.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_21 deleted 2024-04-23T21:51:36.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:36.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:36.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:51:36.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:36.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:36.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:36.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:36.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_21 2024-04-23T21:51:36.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:36.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:36.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:51:36.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_21 2024-04-23T21:51:36.241 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:36.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_21: (2) No such file or directory 2024-04-23T21:51:36.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:36.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:36.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:36.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_21 deleted '' 2024-04-23T21:51:36.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:36.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:36.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_21 2024-04-23T21:51:36.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:36.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:36.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:36.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:36.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_21 2024-04-23T21:51:36.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:36.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:36.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_21 2024-04-23T21:51:36.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_21 2024-04-23T21:51:36.377 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:36.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_21: (2) No such file or directory 2024-04-23T21:51:36.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:36.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:36.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:36.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:36.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:36.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_22 2024-04-23T21:51:36.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_22 deleted 2024-04-23T21:51:36.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:36.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:36.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:51:36.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:36.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:36.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:36.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:36.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_22 2024-04-23T21:51:36.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:36.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:36.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_22 2024-04-23T21:51:36.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_22 2024-04-23T21:51:36.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:36.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_22: (2) No such file or directory 2024-04-23T21:51:36.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:36.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:36.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:36.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_22 deleted '' 2024-04-23T21:51:36.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:36.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:36.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_22 2024-04-23T21:51:36.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:36.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:36.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:36.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:36.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_22 2024-04-23T21:51:36.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:36.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:36.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_22 2024-04-23T21:51:36.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_22 2024-04-23T21:51:36.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:36.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_22: (2) No such file or directory 2024-04-23T21:51:36.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:36.583 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_23 2024-04-23T21:51:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_23 deleted 2024-04-23T21:51:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:51:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_23 2024-04-23T21:51:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:36.584 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:36.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:51:36.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_23 2024-04-23T21:51:36.585 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:36.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_23: (2) No such file or directory 2024-04-23T21:51:36.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:36.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:36.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:36.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_23 deleted '' 2024-04-23T21:51:36.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:36.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:36.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_23 2024-04-23T21:51:36.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:36.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:36.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:36.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:36.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_23 2024-04-23T21:51:36.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:36.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:36.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_23 2024-04-23T21:51:36.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_23 2024-04-23T21:51:36.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:36.751 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_23: (2) No such file or directory 2024-04-23T21:51:36.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:36.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:36.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:36.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:36.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:36.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_24 2024-04-23T21:51:36.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_24 deleted 2024-04-23T21:51:36.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:36.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:36.758 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:51:36.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:36.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:36.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:36.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:36.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_24 2024-04-23T21:51:36.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:36.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:36.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:51:36.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_24 2024-04-23T21:51:36.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:36.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_24: (2) No such file or directory 2024-04-23T21:51:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:36.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:36.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_24 deleted '' 2024-04-23T21:51:36.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:36.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:36.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_24 2024-04-23T21:51:36.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:36.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:36.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:36.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:36.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_24 2024-04-23T21:51:36.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:36.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:36.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_24 2024-04-23T21:51:36.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_24 2024-04-23T21:51:36.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:36.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_24: (2) No such file or directory 2024-04-23T21:51:36.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:36.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:36.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:36.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:36.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:36.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_25 2024-04-23T21:51:36.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_25 deleted 2024-04-23T21:51:36.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:36.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:36.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:51:36.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:36.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:36.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:36.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:36.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_25 2024-04-23T21:51:36.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:36.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:36.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:51:36.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_25 2024-04-23T21:51:36.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:36.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_25: (2) No such file or directory 2024-04-23T21:51:36.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:36.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:36.965 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_25 deleted '' 2024-04-23T21:51:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_25 2024-04-23T21:51:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_25 2024-04-23T21:51:37.067 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:37.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:37.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_25 2024-04-23T21:51:37.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_25 2024-04-23T21:51:37.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:37.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_25: (2) No such file or directory 2024-04-23T21:51:37.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:37.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:37.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:37.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:37.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:37.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_26 2024-04-23T21:51:37.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_26 deleted 2024-04-23T21:51:37.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:37.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:37.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:51:37.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:37.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:37.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:37.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:37.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_26 2024-04-23T21:51:37.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:37.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:37.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:51:37.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_26 2024-04-23T21:51:37.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:37.133 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_26: (2) No such file or directory 2024-04-23T21:51:37.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:37.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:37.136 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:37.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_26 deleted '' 2024-04-23T21:51:37.238 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_26 2024-04-23T21:51:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_26 2024-04-23T21:51:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_26 2024-04-23T21:51:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_26 2024-04-23T21:51:37.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:37.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_26: (2) No such file or directory 2024-04-23T21:51:37.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:37.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:37.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:37.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:37.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:37.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_27 2024-04-23T21:51:37.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_27 deleted 2024-04-23T21:51:37.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:37.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:37.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:51:37.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:37.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:37.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:37.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:37.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_27 2024-04-23T21:51:37.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:37.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:37.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_27 2024-04-23T21:51:37.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_27 2024-04-23T21:51:37.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:37.305 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_27: (2) No such file or directory 2024-04-23T21:51:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:37.308 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:37.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_27 deleted '' 2024-04-23T21:51:37.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:37.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:37.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_27 2024-04-23T21:51:37.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:37.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:37.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:37.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:37.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_27 2024-04-23T21:51:37.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:37.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:37.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_27 2024-04-23T21:51:37.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_27 2024-04-23T21:51:37.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:37.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_27: (2) No such file or directory 2024-04-23T21:51:37.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:37.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:37.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:37.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:37.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_28 2024-04-23T21:51:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_28 deleted 2024-04-23T21:51:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:51:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_28 2024-04-23T21:51:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:51:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_28 2024-04-23T21:51:37.446 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:37.476 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_28: (2) No such file or directory 2024-04-23T21:51:37.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:37.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:37.479 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:37.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_28 deleted '' 2024-04-23T21:51:37.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:37.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:37.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_28 2024-04-23T21:51:37.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:37.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:37.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:37.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:37.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_28 2024-04-23T21:51:37.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:37.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:37.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_28 2024-04-23T21:51:37.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_28 2024-04-23T21:51:37.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:37.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_28: (2) No such file or directory 2024-04-23T21:51:37.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:37.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_29 2024-04-23T21:51:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_29 deleted 2024-04-23T21:51:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:51:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_29 2024-04-23T21:51:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:37.617 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:51:37.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_29 2024-04-23T21:51:37.618 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:37.849 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_29: (2) No such file or directory 2024-04-23T21:51:37.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:37.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:37.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:37.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_29 deleted '' 2024-04-23T21:51:37.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:37.954 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:37.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_29 2024-04-23T21:51:37.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:37.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:37.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:37.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:37.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_29 2024-04-23T21:51:37.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:37.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:37.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_29 2024-04-23T21:51:37.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_29 2024-04-23T21:51:37.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:37.990 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_29: (2) No such file or directory 2024-04-23T21:51:37.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:37.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:37.994 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:37.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:37.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:37.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_30 2024-04-23T21:51:37.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_30 deleted 2024-04-23T21:51:37.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:37.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:37.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:51:37.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:37.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:37.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:37.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:37.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_30 2024-04-23T21:51:37.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:37.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:37.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_30 2024-04-23T21:51:37.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_30 2024-04-23T21:51:37.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:38.026 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_30: (2) No such file or directory 2024-04-23T21:51:38.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:38.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:38.029 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:38.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_30 deleted '' 2024-04-23T21:51:38.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:38.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:38.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_30 2024-04-23T21:51:38.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:38.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:38.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:38.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:38.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_30 2024-04-23T21:51:38.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:38.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:38.131 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_30 2024-04-23T21:51:38.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_30 2024-04-23T21:51:38.132 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:38.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_30: (2) No such file or directory 2024-04-23T21:51:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_31 2024-04-23T21:51:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_31 deleted 2024-04-23T21:51:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:38.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:51:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_31 2024-04-23T21:51:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:51:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_31 2024-04-23T21:51:38.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:38.197 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_31: (2) No such file or directory 2024-04-23T21:51:38.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:38.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:38.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:38.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_31 deleted '' 2024-04-23T21:51:38.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:38.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:38.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_31 2024-04-23T21:51:38.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:38.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:38.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:38.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:38.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_31 2024-04-23T21:51:38.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:38.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:38.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_31 2024-04-23T21:51:38.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_31 2024-04-23T21:51:38.303 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:38.334 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_31: (2) No such file or directory 2024-04-23T21:51:38.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:38.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:38.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:38.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:38.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:38.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_32 2024-04-23T21:51:38.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_32 deleted 2024-04-23T21:51:38.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:38.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:38.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:51:38.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:38.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:38.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:38.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:38.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_32 2024-04-23T21:51:38.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:38.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:38.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:51:38.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_32 2024-04-23T21:51:38.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:38.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_32: (2) No such file or directory 2024-04-23T21:51:38.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:38.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:38.372 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:38.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_32 deleted '' 2024-04-23T21:51:38.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:38.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:38.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_32 2024-04-23T21:51:38.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:38.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:38.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:38.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:38.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_32 2024-04-23T21:51:38.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:38.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:38.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_32 2024-04-23T21:51:38.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_32 2024-04-23T21:51:38.475 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:38.506 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_32: (2) No such file or directory 2024-04-23T21:51:38.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:38.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:38.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:38.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_33 2024-04-23T21:51:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_33 deleted 2024-04-23T21:51:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:51:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_33 2024-04-23T21:51:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:38.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:51:38.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_33 2024-04-23T21:51:38.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:38.540 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_33: (2) No such file or directory 2024-04-23T21:51:38.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:38.543 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:38.544 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:38.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_33 deleted '' 2024-04-23T21:51:38.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:38.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:38.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_33 2024-04-23T21:51:38.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:38.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:38.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:38.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:38.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_33 2024-04-23T21:51:38.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:38.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:38.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_33 2024-04-23T21:51:38.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_33 2024-04-23T21:51:38.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:38.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_33: (2) No such file or directory 2024-04-23T21:51:38.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_34 2024-04-23T21:51:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_34 deleted 2024-04-23T21:51:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:51:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_34 2024-04-23T21:51:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:38.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:38.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:51:38.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_34 2024-04-23T21:51:38.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:38.710 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_34: (2) No such file or directory 2024-04-23T21:51:38.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:38.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:38.713 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:38.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_34 deleted '' 2024-04-23T21:51:38.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:38.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:38.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_34 2024-04-23T21:51:38.815 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:38.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:38.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:38.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:38.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_34 2024-04-23T21:51:38.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:38.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:38.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_34 2024-04-23T21:51:38.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_34 2024-04-23T21:51:38.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:38.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_34: (2) No such file or directory 2024-04-23T21:51:38.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:38.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:38.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:38.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:38.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:38.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_35 2024-04-23T21:51:38.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_35 deleted 2024-04-23T21:51:38.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:38.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:38.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:51:38.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:38.852 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:38.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:38.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:38.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_35 2024-04-23T21:51:38.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:38.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:38.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:51:38.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_35 2024-04-23T21:51:38.853 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:38.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_35: (2) No such file or directory 2024-04-23T21:51:38.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:38.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:38.894 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:38.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_35 deleted '' 2024-04-23T21:51:38.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:38.995 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:38.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_35 2024-04-23T21:51:38.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:38.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:38.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:38.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:38.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_35 2024-04-23T21:51:38.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:38.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:38.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_35 2024-04-23T21:51:38.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_35 2024-04-23T21:51:38.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:39.027 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_35: (2) No such file or directory 2024-04-23T21:51:39.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:39.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:39.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:39.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:39.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:39.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_36 2024-04-23T21:51:39.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_36 deleted 2024-04-23T21:51:39.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:39.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:39.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:51:39.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:39.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:39.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:39.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:39.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_36 2024-04-23T21:51:39.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:39.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:39.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_36 2024-04-23T21:51:39.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_36 2024-04-23T21:51:39.033 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:39.062 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_36: (2) No such file or directory 2024-04-23T21:51:39.065 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:39.066 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:39.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_36 deleted '' 2024-04-23T21:51:39.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:39.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:39.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_36 2024-04-23T21:51:39.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:39.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:39.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:39.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:39.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_36 2024-04-23T21:51:39.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:39.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:39.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_36 2024-04-23T21:51:39.168 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_36 2024-04-23T21:51:39.169 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:39.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_36: (2) No such file or directory 2024-04-23T21:51:39.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:39.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:39.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:39.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:39.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:39.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_37 2024-04-23T21:51:39.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_37 deleted 2024-04-23T21:51:39.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:39.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:39.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:51:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_37 2024-04-23T21:51:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:51:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_37 2024-04-23T21:51:39.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:39.234 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_37: (2) No such file or directory 2024-04-23T21:51:39.236 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:39.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:39.237 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:39.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_37 deleted '' 2024-04-23T21:51:39.338 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:39.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:39.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_37 2024-04-23T21:51:39.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:39.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:39.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:39.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:39.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_37 2024-04-23T21:51:39.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:39.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:39.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_37 2024-04-23T21:51:39.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_37 2024-04-23T21:51:39.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:39.370 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_37: (2) No such file or directory 2024-04-23T21:51:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_38 2024-04-23T21:51:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_38 deleted 2024-04-23T21:51:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:51:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:39.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_38 2024-04-23T21:51:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_38 2024-04-23T21:51:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_38 2024-04-23T21:51:39.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:39.404 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_38: (2) No such file or directory 2024-04-23T21:51:39.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:39.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:39.408 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:39.509 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_38 deleted '' 2024-04-23T21:51:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_38 2024-04-23T21:51:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_38 2024-04-23T21:51:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_38 2024-04-23T21:51:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_38 2024-04-23T21:51:39.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:39.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_38: (2) No such file or directory 2024-04-23T21:51:39.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:39.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:39.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:39.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:39.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:39.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_39 2024-04-23T21:51:39.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_39 deleted 2024-04-23T21:51:39.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:39.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:39.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:51:39.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:39.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:39.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:39.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:39.547 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_39 2024-04-23T21:51:39.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:39.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:39.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:51:39.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_39 2024-04-23T21:51:39.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:39.577 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_39: (2) No such file or directory 2024-04-23T21:51:39.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:39.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:39.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:39.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_39 deleted '' 2024-04-23T21:51:39.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:39.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:39.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_39 2024-04-23T21:51:39.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:39.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:39.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:39.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:39.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_39 2024-04-23T21:51:39.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:39.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:39.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_39 2024-04-23T21:51:39.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_39 2024-04-23T21:51:39.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:39.714 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_39: (2) No such file or directory 2024-04-23T21:51:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_40 2024-04-23T21:51:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_40 deleted 2024-04-23T21:51:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:39.718 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:51:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_40 2024-04-23T21:51:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:51:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_40 2024-04-23T21:51:39.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:39.749 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_40: (2) No such file or directory 2024-04-23T21:51:39.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:39.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:39.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:39.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_40 deleted '' 2024-04-23T21:51:39.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:39.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:39.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_40 2024-04-23T21:51:39.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:39.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:39.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:39.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:39.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_40 2024-04-23T21:51:39.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:39.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:39.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_40 2024-04-23T21:51:39.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_40 2024-04-23T21:51:39.855 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:39.887 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_40: (2) No such file or directory 2024-04-23T21:51:39.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:39.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:39.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:39.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:39.891 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:39.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_41 2024-04-23T21:51:39.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_41 deleted 2024-04-23T21:51:39.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:39.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:39.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:51:39.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:39.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:39.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:39.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:39.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_41 2024-04-23T21:51:39.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:39.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:39.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:51:39.892 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_41 2024-04-23T21:51:39.893 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:39.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_41: (2) No such file or directory 2024-04-23T21:51:39.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:39.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:39.929 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:40.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_41 deleted '' 2024-04-23T21:51:40.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:40.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:40.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_41 2024-04-23T21:51:40.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:40.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:40.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:40.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:40.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_41 2024-04-23T21:51:40.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:40.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:40.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_41 2024-04-23T21:51:40.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_41 2024-04-23T21:51:40.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:40.064 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_41: (2) No such file or directory 2024-04-23T21:51:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_42 2024-04-23T21:51:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_42 deleted 2024-04-23T21:51:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:51:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:40.068 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:40.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_42 2024-04-23T21:51:40.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:40.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:40.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_42 2024-04-23T21:51:40.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_42 2024-04-23T21:51:40.069 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:40.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_42: (2) No such file or directory 2024-04-23T21:51:40.101 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:40.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:40.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:40.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_42 deleted '' 2024-04-23T21:51:40.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:40.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:40.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_42 2024-04-23T21:51:40.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:40.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:40.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:40.203 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:40.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_42 2024-04-23T21:51:40.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:40.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:40.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_42 2024-04-23T21:51:40.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_42 2024-04-23T21:51:40.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:40.235 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_42: (2) No such file or directory 2024-04-23T21:51:40.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:40.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:40.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:40.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:40.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:40.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_43 2024-04-23T21:51:40.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_43 deleted 2024-04-23T21:51:40.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:40.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:40.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:51:40.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:40.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:40.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:40.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:40.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_43 2024-04-23T21:51:40.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:40.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:40.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:51:40.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_43 2024-04-23T21:51:40.240 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:40.270 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_43: (2) No such file or directory 2024-04-23T21:51:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:40.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:40.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_43 deleted '' 2024-04-23T21:51:40.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:40.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:40.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_43 2024-04-23T21:51:40.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:40.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:40.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:40.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:40.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_43 2024-04-23T21:51:40.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:40.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:40.375 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_43 2024-04-23T21:51:40.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_43 2024-04-23T21:51:40.376 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:40.406 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_43: (2) No such file or directory 2024-04-23T21:51:40.410 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:40.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:40.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:40.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:40.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:40.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_44 2024-04-23T21:51:40.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_44 deleted 2024-04-23T21:51:40.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:40.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:40.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:51:40.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:40.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:40.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:40.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:40.411 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_44 2024-04-23T21:51:40.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:40.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:40.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:51:40.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_44 2024-04-23T21:51:40.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:40.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_44: (2) No such file or directory 2024-04-23T21:51:40.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:40.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:40.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:40.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_44 deleted '' 2024-04-23T21:51:40.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:40.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:40.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_44 2024-04-23T21:51:40.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:40.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:40.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:40.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:40.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_44 2024-04-23T21:51:40.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:40.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:40.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_44 2024-04-23T21:51:40.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_44 2024-04-23T21:51:40.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:40.576 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_44: (2) No such file or directory 2024-04-23T21:51:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_45 2024-04-23T21:51:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_45 deleted 2024-04-23T21:51:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:51:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:40.581 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:40.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:40.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:40.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_45 2024-04-23T21:51:40.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:40.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:40.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:51:40.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_45 2024-04-23T21:51:40.582 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:40.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_45: (2) No such file or directory 2024-04-23T21:51:40.614 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:40.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:40.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:40.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_45 deleted '' 2024-04-23T21:51:40.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:40.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:40.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_45 2024-04-23T21:51:40.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:40.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:40.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:40.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:40.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_45 2024-04-23T21:51:40.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:40.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:40.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_45 2024-04-23T21:51:40.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_45 2024-04-23T21:51:40.717 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:40.748 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_45: (2) No such file or directory 2024-04-23T21:51:40.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:40.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:40.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:40.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:40.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:40.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_46 2024-04-23T21:51:40.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_46 deleted 2024-04-23T21:51:40.752 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:40.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:40.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:51:40.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:40.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:40.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:40.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:40.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_46 2024-04-23T21:51:40.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:40.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:40.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:51:40.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_46 2024-04-23T21:51:40.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:40.783 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_46: (2) No such file or directory 2024-04-23T21:51:40.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:40.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:40.786 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:40.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_46 deleted '' 2024-04-23T21:51:40.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:40.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:40.888 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_46 2024-04-23T21:51:40.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:40.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:40.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:40.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:40.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_46 2024-04-23T21:51:40.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:40.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:40.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_46 2024-04-23T21:51:40.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_46 2024-04-23T21:51:40.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:40.926 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_46: (2) No such file or directory 2024-04-23T21:51:40.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:40.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:40.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:40.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:40.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:40.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_47 2024-04-23T21:51:40.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_47 deleted 2024-04-23T21:51:40.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:40.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:40.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:51:40.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:40.931 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:40.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:40.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:40.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_47 2024-04-23T21:51:40.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:40.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:40.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:51:40.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_47 2024-04-23T21:51:40.932 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:40.973 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_47: (2) No such file or directory 2024-04-23T21:51:40.976 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:40.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:40.977 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:41.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_47 deleted '' 2024-04-23T21:51:41.078 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_47 2024-04-23T21:51:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_47 2024-04-23T21:51:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_47 2024-04-23T21:51:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_47 2024-04-23T21:51:41.079 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:41.109 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_47: (2) No such file or directory 2024-04-23T21:51:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_48 2024-04-23T21:51:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_48 deleted 2024-04-23T21:51:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:41.114 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:51:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_48 2024-04-23T21:51:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:51:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_48 2024-04-23T21:51:41.115 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:41.145 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_48: (2) No such file or directory 2024-04-23T21:51:41.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:41.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:41.148 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_48 deleted '' 2024-04-23T21:51:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_48 2024-04-23T21:51:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_48 2024-04-23T21:51:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_48 2024-04-23T21:51:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_48 2024-04-23T21:51:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:41.281 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_48: (2) No such file or directory 2024-04-23T21:51:41.285 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_49 2024-04-23T21:51:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_49 deleted 2024-04-23T21:51:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:51:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:41.286 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:41.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_49 2024-04-23T21:51:41.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:41.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:41.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_49 2024-04-23T21:51:41.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_49 2024-04-23T21:51:41.287 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:41.316 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_49: (2) No such file or directory 2024-04-23T21:51:41.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:41.319 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:41.320 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:41.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_49 deleted '' 2024-04-23T21:51:41.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:41.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:41.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_49 2024-04-23T21:51:41.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:41.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:41.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:41.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:41.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_49 2024-04-23T21:51:41.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:41.422 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:41.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_49 2024-04-23T21:51:41.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_49 2024-04-23T21:51:41.423 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:41.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_49: (2) No such file or directory 2024-04-23T21:51:41.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:41.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:41.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:41.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:41.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2024-04-23T21:51:41.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_50 2024-04-23T21:51:41.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_50 deleted 2024-04-23T21:51:41.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:41.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:41.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:51:41.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:51:41.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:41.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:41.458 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:41.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_50 2024-04-23T21:51:41.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:41.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:41.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:51:41.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_50 2024-04-23T21:51:41.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:41.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_50: (2) No such file or directory 2024-04-23T21:51:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:51:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:41.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_50 deleted '' 2024-04-23T21:51:41.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:41.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:41.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_50 2024-04-23T21:51:41.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:51:41.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:41.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:41.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:41.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_50 2024-04-23T21:51:41.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:41.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:41.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_50 2024-04-23T21:51:41.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_50 2024-04-23T21:51:41.594 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:41.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_50: (2) No such file or directory 2024-04-23T21:51:41.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:51:41.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:41.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:51:41.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:41.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ testlog 'TEST: delete images during bootstrap' 2024-04-23T21:51:41.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ tee -a /tmp/tmp.8Tyc9aSFBO/rbd-mirror.test.log 2024-04-23T21:51:41.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ date '+%F %T' 2024-04-23T21:51:41.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo 2024-04-23 21:51:41 TEST: delete images during bootstrap 2024-04-23T21:51:41.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:2024-04-23 21:51:41 TEST: delete images during bootstrap 2024-04-23T21:51:41.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_pool_mirror_mode cluster1 mirror image 2024-04-23T21:51:41.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:41.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:41.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=image 2024-04-23T21:51:41.635 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster1 mirror pool enable mirror image 2024-04-23T21:51:41.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:note: changing mirroring mode from pool to image 2024-04-23T21:51:41.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_pool_mirror_mode cluster2 mirror image 2024-04-23T21:51:41.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:41.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:41.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=image 2024-04-23T21:51:41.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror pool enable mirror image 2024-04-23T21:51:41.711 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:note: changing mirroring mode from pool to image 2024-04-23T21:51:41.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ start_mirror cluster1 2024-04-23T21:51:41.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:41.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance 2024-04-23T21:51:41.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set_cluster_instance cluster1 cluster instance 2024-04-23T21:51:41.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local val=cluster1 2024-04-23T21:51:41.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster_var_name=cluster 2024-04-23T21:51:41.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local instance_var_name=instance 2024-04-23T21:51:41.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cluster=cluster1 2024-04-23T21:51:41.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=cluster1 2024-04-23T21:51:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' cluster1 = cluster1 ']' 2024-04-23T21:51:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ instance=0 2024-04-23T21:51:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval cluster=cluster1 2024-04-23T21:51:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ cluster=cluster1 2024-04-23T21:51:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ eval instance=0 2024-04-23T21:51:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ instance=0 2024-04-23T21:51:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:51:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:51:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=test 2024-04-23T21:51:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 1 10 2024-04-23T21:51:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:51:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_1 2024-04-23T21:51:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_1 512M 2024-04-23T21:51:41.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:41.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:51:41.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:41.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:51:41.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:51:41.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:51:41.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:51:41.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 512M ']' 2024-04-23T21:51:41.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=512M 2024-04-23T21:51:41.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:51:41.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering,exclusive-lock,journaling mirror/image_1 2024-04-23T21:51:41.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ enable_mirror cluster2 mirror image_1 2024-04-23T21:51:41.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:41.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:41.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:51:41.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:51:41.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_1 journal 2024-04-23T21:51:41.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring enabled 2024-04-23T21:51:41.827 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 info mirror/image_1 2024-04-23T21:51:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_1': 2024-04-23T21:51:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:51:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:51:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 0 2024-04-23T21:51:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 2671402c53fb 2024-04-23T21:51:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.2671402c53fb 2024-04-23T21:51:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:51:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:51:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:51:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:51:41.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:51:41 2024 2024-04-23T21:51:41.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:51:41 2024 2024-04-23T21:51:41.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:51:41 2024 2024-04-23T21:51:41.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 2671402c53fb 2024-04-23T21:51:41.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:51:41.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:51:41.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: fa732796-5839-4536-b1aa-5e433bfb5d4b 2024-04-23T21:51:41.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: true 2024-04-23T21:51:41.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_1 2024-04-23T21:51:41.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:41.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:41.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:51:41.867 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:51:41.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=14 2024-04-23T21:51:41.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:51:41.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_1 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:51:41.908 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3005837160246656) > weight (0.25) 2024-04-23T21:51:41.908 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2024-04-23T21:51:41.909 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2024-04-23T21:51:41.909 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2024-04-23T21:51:41.909 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2024-04-23T21:51:41.909 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:51:41.910 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:51:41.909+0000 7f46e4b02640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 60044) UID: 1000 2024-04-23T21:51:44.967 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 51619840 bytes 2024-04-23T21:51:45.531 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 58591232 bytes 2024-04-23T21:51:47.580 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 106708992 bytes 2024-04-23T21:51:48.012 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2024-04-23T21:51:48.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2024-04-23T21:51:48.012 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2024-04-23T21:51:48.015 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2024-04-23T21:51:48.015 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2024-04-23T21:51:49.667 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 148172800 bytes 2024-04-23T21:51:52.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 190871552 bytes 2024-04-23T21:51:53.239 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:600 IOs, 207039488 bytes 2024-04-23T21:51:54.121 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:700 IOs, 214334464 bytes 2024-04-23T21:51:54.415 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:800 IOs, 215829504 bytes 2024-04-23T21:51:55.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:51:55.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:51:55.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:51:55.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:55.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_1 present 2024-04-23T21:51:55.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:55.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:55.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:51:55.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=present 2024-04-23T21:51:55.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:51:55.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:51:55.880 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:51:55.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_1 2024-04-23T21:51:55.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:55.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:55.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_1 2024-04-23T21:51:55.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_1 2024-04-23T21:51:55.881 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:55.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id=121dc5cc41f3 2024-04-23T21:51:55.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:51:55.930 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:51:56.031 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_1 present 121dc5cc41f3 2024-04-23T21:51:56.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:51:56.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:56.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:51:56.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=present 2024-04-23T21:51:56.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id=121dc5cc41f3 2024-04-23T21:51:56.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:51:56.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:51:56.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_1 2024-04-23T21:51:56.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:51:56.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:51:56.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_1 2024-04-23T21:51:56.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_1 2024-04-23T21:51:56.032 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:51:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id=121dc5cc41f3 2024-04-23T21:51:56.086 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 121dc5cc41f3 2024-04-23T21:51:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -z 121dc5cc41f3 -o 121dc5cc41f3 = 121dc5cc41f3 2024-04-23T21:51:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=present 2024-04-23T21:51:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test present = present 2024-04-23T21:51:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:51:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ disable_mirror cluster2 mirror image_1 2024-04-23T21:51:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:51:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:51:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:51:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_1 2024-04-23T21:51:56.228 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi150.stderr:2024-04-23T21:51:56.228+0000 7fe32be48640 -1 rbd::mirror::ImageMap: 0x555848bc9320 handle_peer_ack_remove: failed to remove global_image_id=fa732796-5839-4536-b1aa-5e433bfb5d4b 2024-04-23T21:52:25.782 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring disabled 2024-04-23T21:52:25.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_1 deleted 2024-04-23T21:52:25.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:52:25.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:52:25.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:52:25.790 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:52:25.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:52:25.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:52:25.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:52:25.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_1 2024-04-23T21:52:25.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:52:25.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:52:25.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_1 2024-04-23T21:52:25.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_1 2024-04-23T21:52:25.791 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:52:25.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_1: (2) No such file or directory 2024-04-23T21:52:25.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:52:25.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:52:25.825 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:52:25.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_1 deleted '' 2024-04-23T21:52:25.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:52:25.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:52:25.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:52:25.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:52:25.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:52:25.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:52:25.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:52:25.927 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_1 2024-04-23T21:52:25.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:52:25.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:52:25.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_1 2024-04-23T21:52:25.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_1 2024-04-23T21:52:25.928 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:52:25.958 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_1: (2) No such file or directory 2024-04-23T21:52:25.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:52:25.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:52:25.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:52:25.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:52:25.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ purge_snapshots cluster2 mirror image_1 2024-04-23T21:52:25.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:52:25.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:52:25.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:52:25.962 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap purge mirror/image_1 2024-04-23T21:52:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_1 2024-04-23T21:52:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:52:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:52:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:52:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:52:26.004 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:52:26.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_1 2024-04-23T21:52:26.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:52:26.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:52:26.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_1 2024-04-23T21:52:26.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_1 2024-04-23T21:52:26.047 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_1 2024-04-23T21:52:26.290 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-04-23T21:52:26.293 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:52:26.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:52:26.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_2 2024-04-23T21:52:26.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_2 512M 2024-04-23T21:52:26.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:52:26.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:52:26.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:52:26.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:52:26.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:52:26.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:52:26.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:52:26.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 512M ']' 2024-04-23T21:52:26.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=512M 2024-04-23T21:52:26.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:52:26.295 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering,exclusive-lock,journaling mirror/image_2 2024-04-23T21:52:26.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ enable_mirror cluster2 mirror image_2 2024-04-23T21:52:26.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:52:26.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:52:26.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:52:26.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:52:26.348 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_2 journal 2024-04-23T21:52:26.395 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring enabled 2024-04-23T21:52:26.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 info mirror/image_2 2024-04-23T21:52:26.436 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_2': 2024-04-23T21:52:26.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:52:26.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:52:26.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 0 2024-04-23T21:52:26.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 26935650f6 2024-04-23T21:52:26.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.26935650f6 2024-04-23T21:52:26.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:52:26.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:52:26.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:52:26.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:52:26.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:52:26 2024 2024-04-23T21:52:26.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:52:26 2024 2024-04-23T21:52:26.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:52:26 2024 2024-04-23T21:52:26.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 26935650f6 2024-04-23T21:52:26.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:52:26.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:52:26.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 61ad1304-e797-448d-b807-572e8a8916e4 2024-04-23T21:52:26.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: true 2024-04-23T21:52:26.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_2 2024-04-23T21:52:26.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:52:26.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:52:26.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:52:26.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:52:26.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=9 2024-04-23T21:52:26.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:52:26.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_2 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:52:29.535 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 48910336 bytes 2024-04-23T21:52:30.889 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 116194304 bytes 2024-04-23T21:52:32.339 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 116832256 bytes 2024-04-23T21:52:32.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 118372352 bytes 2024-04-23T21:52:35.302 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 169226240 bytes 2024-04-23T21:52:35.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:52:35.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:52:35.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:52:35.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:52:35.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_2 present 2024-04-23T21:52:35.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:52:35.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:52:35.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:52:35.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=present 2024-04-23T21:52:35.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:52:35.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:52:35.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:52:35.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_2 2024-04-23T21:52:35.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:52:35.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:52:35.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:52:35.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_2 2024-04-23T21:52:35.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:52:35.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id=121d394d9a80 2024-04-23T21:52:35.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:52:35.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:52:35.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_2 present 121d394d9a80 2024-04-23T21:52:35.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:52:35.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:52:35.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:52:35.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=present 2024-04-23T21:52:35.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id=121d394d9a80 2024-04-23T21:52:35.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:52:35.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:52:35.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_2 2024-04-23T21:52:35.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:52:35.603 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:52:35.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:52:35.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_2 2024-04-23T21:52:35.604 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:52:35.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id=121d394d9a80 2024-04-23T21:52:35.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 121d394d9a80 2024-04-23T21:52:35.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -z 121d394d9a80 -o 121d394d9a80 = 121d394d9a80 2024-04-23T21:52:35.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=present 2024-04-23T21:52:35.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test present = present 2024-04-23T21:52:35.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:52:35.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ disable_mirror cluster2 mirror image_2 2024-04-23T21:52:35.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:52:35.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:52:35.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:52:35.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_2 2024-04-23T21:52:41.414 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5100069304773093) > weight (0.25) 2024-04-23T21:52:41.414 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8579180361301579) > weight (0.25) 2024-04-23T21:52:41.414 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8161078117172748) > weight (0.25) 2024-04-23T21:52:41.414 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2024-04-23T21:52:41.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2024-04-23T21:52:41.415 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2024-04-23T21:52:41.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:52:41.415+0000 7fdb900ca640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 97510) UID: 1000 2024-04-23T21:52:46.416 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2024-04-23T21:52:46.416 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:52:46.416 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2024-04-23T21:52:46.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2024-04-23T21:52:46.416 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2024-04-23T21:52:46.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2024-04-23T21:52:46.420 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2024-04-23T21:53:05.545 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring disabled 2024-04-23T21:53:05.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_2 deleted 2024-04-23T21:53:05.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:53:05.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:05.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:53:05.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:53:05.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:53:05.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:53:05.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:53:05.557 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_2 2024-04-23T21:53:05.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:53:05.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:53:05.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:53:05.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_2 2024-04-23T21:53:05.558 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:53:05.590 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_2: (2) No such file or directory 2024-04-23T21:53:05.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:53:05.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:53:05.593 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:53:05.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_2 deleted '' 2024-04-23T21:53:05.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:53:05.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:05.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:53:05.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:53:05.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:53:05.695 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:53:05.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:53:05.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_2 2024-04-23T21:53:05.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:53:05.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:53:05.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_2 2024-04-23T21:53:05.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_2 2024-04-23T21:53:05.696 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:53:05.728 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_2: (2) No such file or directory 2024-04-23T21:53:05.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:53:05.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:53:05.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:53:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:53:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ purge_snapshots cluster2 mirror image_2 2024-04-23T21:53:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:53:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:53:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap purge mirror/image_2 2024-04-23T21:53:05.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_2 2024-04-23T21:53:05.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:53:05.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:05.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:53:05.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:53:05.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:53:05.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_2 2024-04-23T21:53:05.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:53:05.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:05.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_2 2024-04-23T21:53:05.775 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_2 2024-04-23T21:53:05.817 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_2 2024-04-23T21:53:06.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-04-23T21:53:06.042 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:53:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:53:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_3 2024-04-23T21:53:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_3 512M 2024-04-23T21:53:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:53:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:53:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:53:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:53:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:53:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:53:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 512M ']' 2024-04-23T21:53:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=512M 2024-04-23T21:53:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:53:06.043 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering,exclusive-lock,journaling mirror/image_3 2024-04-23T21:53:06.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ enable_mirror cluster2 mirror image_3 2024-04-23T21:53:06.097 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:53:06.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:06.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:53:06.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:53:06.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_3 journal 2024-04-23T21:53:06.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring enabled 2024-04-23T21:53:06.150 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 info mirror/image_3 2024-04-23T21:53:06.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_3': 2024-04-23T21:53:06.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:53:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:53:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 0 2024-04-23T21:53:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 26b0dc411c77 2024-04-23T21:53:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.26b0dc411c77 2024-04-23T21:53:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:53:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:53:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:53:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:53:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:53:06 2024 2024-04-23T21:53:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:53:06 2024 2024-04-23T21:53:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:53:06 2024 2024-04-23T21:53:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 26b0dc411c77 2024-04-23T21:53:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:53:06.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:53:06.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 8d2a3d56-33ea-439c-a236-b7ec9a09b9a5 2024-04-23T21:53:06.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: true 2024-04-23T21:53:06.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_3 2024-04-23T21:53:06.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:53:06.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:06.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:53:06.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:53:06.190 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=10 2024-04-23T21:53:06.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:53:06.191 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_3 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:53:07.731 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 22422528 bytes 2024-04-23T21:53:08.017 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8931878066571254) > weight (0.25) 2024-04-23T21:53:08.017 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2024-04-23T21:53:08.017 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2024-04-23T21:53:08.017 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2024-04-23T21:53:08.019 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:53:08.018+0000 7efd81f8e640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 103331) UID: 1000 2024-04-23T21:53:09.433 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 44386304 bytes 2024-04-23T21:53:10.282 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 54765568 bytes 2024-04-23T21:53:12.189 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 99452928 bytes 2024-04-23T21:53:15.152 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 128453632 bytes 2024-04-23T21:53:16.050 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:600 IOs, 152093696 bytes 2024-04-23T21:53:16.198 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:53:16.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:53:16.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:53:16.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:53:16.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_3 present 2024-04-23T21:53:16.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:53:16.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:16.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:53:16.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=present 2024-04-23T21:53:16.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:53:16.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:53:16.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:53:16.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_3 2024-04-23T21:53:16.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:53:16.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:53:16.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:53:16.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_3 2024-04-23T21:53:16.200 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:53:16.242 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id=121d2f75f504 2024-04-23T21:53:16.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:53:16.243 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:53:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_3 present 121d2f75f504 2024-04-23T21:53:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:53:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:53:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=present 2024-04-23T21:53:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id=121d2f75f504 2024-04-23T21:53:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:53:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:53:16.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_3 2024-04-23T21:53:16.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:53:16.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:53:16.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:53:16.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_3 2024-04-23T21:53:16.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:53:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id=121d2f75f504 2024-04-23T21:53:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 121d2f75f504 2024-04-23T21:53:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -z 121d2f75f504 -o 121d2f75f504 = 121d2f75f504 2024-04-23T21:53:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=present 2024-04-23T21:53:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test present = present 2024-04-23T21:53:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:53:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ disable_mirror cluster2 mirror image_3 2024-04-23T21:53:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:53:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:53:16.388 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_3 2024-04-23T21:53:17.018 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2024-04-23T21:53:17.018 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:53:17.019 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2024-04-23T21:53:17.019 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2024-04-23T21:53:17.019 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2024-04-23T21:53:17.022 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2024-04-23T21:53:17.022 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2024-04-23T21:53:46.621 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring disabled 2024-04-23T21:53:46.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_3 deleted 2024-04-23T21:53:46.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:53:46.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:46.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:53:46.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:53:46.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:53:46.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:53:46.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:53:46.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_3 2024-04-23T21:53:46.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:53:46.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:53:46.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:53:46.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_3 2024-04-23T21:53:46.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:53:46.663 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_3: (2) No such file or directory 2024-04-23T21:53:46.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:53:46.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:53:46.666 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:53:46.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_3 deleted '' 2024-04-23T21:53:46.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:53:46.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:46.768 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:53:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:53:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:53:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:53:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:53:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_3 2024-04-23T21:53:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:53:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:53:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_3 2024-04-23T21:53:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_3 2024-04-23T21:53:46.769 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:53:46.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_3: (2) No such file or directory 2024-04-23T21:53:46.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:53:46.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:53:46.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:53:46.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:53:46.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ purge_snapshots cluster2 mirror image_3 2024-04-23T21:53:46.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:53:46.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:46.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:53:46.804 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap purge mirror/image_3 2024-04-23T21:53:46.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_3 2024-04-23T21:53:46.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:53:46.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:46.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:53:46.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:53:46.847 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:53:46.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_3 2024-04-23T21:53:46.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:53:46.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:46.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_3 2024-04-23T21:53:46.848 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_3 2024-04-23T21:53:46.890 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_3 2024-04-23T21:53:47.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-04-23T21:53:47.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:53:47.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:53:47.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_4 2024-04-23T21:53:47.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_4 512M 2024-04-23T21:53:47.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:53:47.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:53:47.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:47.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:53:47.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:53:47.111 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:53:47.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:53:47.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 512M ']' 2024-04-23T21:53:47.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=512M 2024-04-23T21:53:47.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:53:47.112 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering,exclusive-lock,journaling mirror/image_4 2024-04-23T21:53:47.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ enable_mirror cluster2 mirror image_4 2024-04-23T21:53:47.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:53:47.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:47.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:53:47.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:53:47.167 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_4 journal 2024-04-23T21:53:47.213 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring enabled 2024-04-23T21:53:47.219 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 info mirror/image_4 2024-04-23T21:53:47.255 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_4': 2024-04-23T21:53:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:53:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:53:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 0 2024-04-23T21:53:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 26cfb196bc64 2024-04-23T21:53:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.26cfb196bc64 2024-04-23T21:53:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:53:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:53:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:53:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:53:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:53:47 2024 2024-04-23T21:53:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:53:47 2024 2024-04-23T21:53:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:53:47 2024 2024-04-23T21:53:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 26cfb196bc64 2024-04-23T21:53:47.256 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:53:47.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:53:47.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: b67bac1a-4d24-4273-8385-935d2a18db65 2024-04-23T21:53:47.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: true 2024-04-23T21:53:47.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_4 2024-04-23T21:53:47.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:53:47.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:47.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:53:47.259 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:53:47.260 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=12 2024-04-23T21:53:47.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:53:47.261 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_4 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:53:47.671 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 23694336 bytes 2024-04-23T21:53:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 76274688 bytes 2024-04-23T21:53:53.759 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 98960384 bytes 2024-04-23T21:53:54.203 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:53:54.203+0000 7f83ee35e640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 6 2024-04-23T21:53:54.207 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:53:54.205+0000 7f83eb358640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 11 2024-04-23T21:53:54.207 INFO:tasks.ceph.cluster2.osd.0.smithi150.stderr:2024-04-23T21:53:54.206+0000 7f83ebb59640 -1 bdev(0x562a4d6ace00 /var/lib/ceph/osd/cluster2-0/block) aio_submit retries 6 2024-04-23T21:53:54.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 116023296 bytes 2024-04-23T21:53:56.093 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 149429248 bytes 2024-04-23T21:53:59.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:600 IOs, 205215744 bytes 2024-04-23T21:53:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:53:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:53:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:53:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:53:59.273 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_4 present 2024-04-23T21:53:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:53:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:53:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=present 2024-04-23T21:53:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:53:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:53:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:53:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_4 2024-04-23T21:53:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:53:59.274 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:53:59.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:53:59.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_4 2024-04-23T21:53:59.275 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:53:59.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id=121d5e784cb1 2024-04-23T21:53:59.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:53:59.318 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:53:59.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_4 present 121d5e784cb1 2024-04-23T21:53:59.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:53:59.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:59.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:53:59.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=present 2024-04-23T21:53:59.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id=121d5e784cb1 2024-04-23T21:53:59.420 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:53:59.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:53:59.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_4 2024-04-23T21:53:59.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:53:59.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:53:59.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:53:59.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_4 2024-04-23T21:53:59.421 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:53:59.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id=121d5e784cb1 2024-04-23T21:53:59.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 121d5e784cb1 2024-04-23T21:53:59.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -z 121d5e784cb1 -o 121d5e784cb1 = 121d5e784cb1 2024-04-23T21:53:59.463 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=present 2024-04-23T21:53:59.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test present = present 2024-04-23T21:53:59.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:53:59.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ disable_mirror cluster2 mirror image_4 2024-04-23T21:53:59.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:53:59.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:53:59.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:53:59.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_4 2024-04-23T21:54:29.597 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring disabled 2024-04-23T21:54:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_4 deleted 2024-04-23T21:54:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:54:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:54:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:54:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:54:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:54:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:54:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:54:29.605 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_4 2024-04-23T21:54:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:54:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:54:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:54:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_4 2024-04-23T21:54:29.606 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:54:29.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_4: (2) No such file or directory 2024-04-23T21:54:29.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:54:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:54:29.643 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:54:29.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_4 deleted '' 2024-04-23T21:54:29.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:54:29.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:54:29.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:54:29.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:54:29.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:54:29.743 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:54:29.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:54:29.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_4 2024-04-23T21:54:29.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:54:29.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:54:29.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_4 2024-04-23T21:54:29.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_4 2024-04-23T21:54:29.744 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:54:29.774 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_4: (2) No such file or directory 2024-04-23T21:54:29.777 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:54:29.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:54:29.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:54:29.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:54:29.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ purge_snapshots cluster2 mirror image_4 2024-04-23T21:54:29.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:54:29.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:54:29.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:54:29.778 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap purge mirror/image_4 2024-04-23T21:54:29.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_4 2024-04-23T21:54:29.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:54:29.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:54:29.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:54:29.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:54:29.820 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:54:29.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_4 2024-04-23T21:54:29.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:54:29.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:54:29.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_4 2024-04-23T21:54:29.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_4 2024-04-23T21:54:29.864 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_4 2024-04-23T21:54:30.090 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-04-23T21:54:30.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:54:30.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:54:30.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_5 2024-04-23T21:54:30.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_5 512M 2024-04-23T21:54:30.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:54:30.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:54:30.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:54:30.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:54:30.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:54:30.095 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:54:30.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:54:30.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 512M ']' 2024-04-23T21:54:30.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=512M 2024-04-23T21:54:30.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:54:30.096 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering,exclusive-lock,journaling mirror/image_5 2024-04-23T21:54:30.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ enable_mirror cluster2 mirror image_5 2024-04-23T21:54:30.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:54:30.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:54:30.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:54:30.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:54:30.149 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_5 journal 2024-04-23T21:54:30.194 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring enabled 2024-04-23T21:54:30.199 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 info mirror/image_5 2024-04-23T21:54:30.437 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_5': 2024-04-23T21:54:30.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:54:30.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:54:30.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 0 2024-04-23T21:54:30.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 26e21ac23603 2024-04-23T21:54:30.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.26e21ac23603 2024-04-23T21:54:30.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:54:30.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:54:30.438 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:54:30.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:54:30.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:54:30 2024 2024-04-23T21:54:30.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:54:30 2024 2024-04-23T21:54:30.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:54:30 2024 2024-04-23T21:54:30.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 26e21ac23603 2024-04-23T21:54:30.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:54:30.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:54:30.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: fd0c9899-4dc0-4d2d-bff1-50f2fd9cea3a 2024-04-23T21:54:30.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: true 2024-04-23T21:54:30.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_5 2024-04-23T21:54:30.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:54:30.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:54:30.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:54:30.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:54:30.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=13 2024-04-23T21:54:30.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:54:30.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_5 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:54:31.420 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6498001099166067) > weight (0.25) 2024-04-23T21:54:31.421 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9780613252650499) > weight (0.25) 2024-04-23T21:54:31.421 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2024-04-23T21:54:31.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2024-04-23T21:54:31.421 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2024-04-23T21:54:31.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:54:31.421+0000 7f33d459d640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 91925) UID: 1000 2024-04-23T21:54:31.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 21328896 bytes 2024-04-23T21:54:33.669 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 38832128 bytes 2024-04-23T21:54:34.369 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 60096512 bytes 2024-04-23T21:54:35.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:54:35.388+0000 7f99a2eb5640 -1 rbd::mirror::image_replayer::journal::Replayer: 0x5619a9928380 get_replay_status: replay not running 2024-04-23T21:54:36.422 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2024-04-23T21:54:36.422 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:54:36.423 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2024-04-23T21:54:36.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2024-04-23T21:54:36.423 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2024-04-23T21:54:36.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2024-04-23T21:54:36.426 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2024-04-23T21:54:37.803 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 125635584 bytes 2024-04-23T21:54:38.419 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 137666560 bytes 2024-04-23T21:54:40.441 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:600 IOs, 169704448 bytes 2024-04-23T21:54:42.382 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:700 IOs, 219340800 bytes 2024-04-23T21:54:43.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:54:43.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:54:43.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:54:43.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:54:43.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_5 present 2024-04-23T21:54:43.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:54:43.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:54:43.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:54:43.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=present 2024-04-23T21:54:43.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:54:43.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:54:43.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:54:43.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_5 2024-04-23T21:54:43.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:54:43.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:54:43.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:54:43.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_5 2024-04-23T21:54:43.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:54:43.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id=121dc1841315 2024-04-23T21:54:43.499 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:54:43.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:54:43.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_5 present 121dc1841315 2024-04-23T21:54:43.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:54:43.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:54:43.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:54:43.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=present 2024-04-23T21:54:43.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id=121dc1841315 2024-04-23T21:54:43.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:54:43.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:54:43.601 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_5 2024-04-23T21:54:43.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:54:43.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:54:43.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:54:43.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_5 2024-04-23T21:54:43.602 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:54:43.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id=121dc1841315 2024-04-23T21:54:43.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 121dc1841315 2024-04-23T21:54:43.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -z 121dc1841315 -o 121dc1841315 = 121dc1841315 2024-04-23T21:54:43.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=present 2024-04-23T21:54:43.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test present = present 2024-04-23T21:54:43.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:54:43.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ disable_mirror cluster2 mirror image_5 2024-04-23T21:54:43.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:54:43.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:54:43.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:54:43.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_5 2024-04-23T21:55:13.675 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring disabled 2024-04-23T21:55:13.683 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_5 deleted 2024-04-23T21:55:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:55:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:55:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:55:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:55:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:55:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:55:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_5 2024-04-23T21:55:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:55:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:55:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:55:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_5 2024-04-23T21:55:13.685 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:55:13.716 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_5: (2) No such file or directory 2024-04-23T21:55:13.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:55:13.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:55:13.719 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:55:13.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_5 deleted '' 2024-04-23T21:55:13.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:55:13.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:13.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:55:13.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:55:13.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:55:13.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:55:13.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:55:13.821 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_5 2024-04-23T21:55:13.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:55:13.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:55:13.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_5 2024-04-23T21:55:13.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_5 2024-04-23T21:55:13.822 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:55:13.854 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_5: (2) No such file or directory 2024-04-23T21:55:13.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:55:13.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:55:13.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:55:13.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:55:13.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ purge_snapshots cluster2 mirror image_5 2024-04-23T21:55:13.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:55:13.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:13.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:55:13.858 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap purge mirror/image_5 2024-04-23T21:55:13.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_5 2024-04-23T21:55:13.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:55:13.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:13.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:55:13.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:55:13.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:55:13.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_5 2024-04-23T21:55:13.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:55:13.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:13.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_5 2024-04-23T21:55:13.901 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_5 2024-04-23T21:55:13.946 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_5 2024-04-23T21:55:14.162 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-04-23T21:55:14.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:55:14.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:55:14.165 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_6 2024-04-23T21:55:14.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_6 512M 2024-04-23T21:55:14.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:55:14.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:55:14.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:14.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:55:14.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:55:14.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:55:14.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:55:14.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 512M ']' 2024-04-23T21:55:14.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=512M 2024-04-23T21:55:14.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:55:14.166 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering,exclusive-lock,journaling mirror/image_6 2024-04-23T21:55:14.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ enable_mirror cluster2 mirror image_6 2024-04-23T21:55:14.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:55:14.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:14.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:55:14.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:55:14.220 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_6 journal 2024-04-23T21:55:14.469 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring enabled 2024-04-23T21:55:14.474 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 info mirror/image_6 2024-04-23T21:55:14.511 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_6': 2024-04-23T21:55:14.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:55:14.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:55:14.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 0 2024-04-23T21:55:14.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 26f94cd88cfe 2024-04-23T21:55:14.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.26f94cd88cfe 2024-04-23T21:55:14.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:55:14.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:55:14.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:55:14.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:55:14.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:55:14 2024 2024-04-23T21:55:14.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:55:14 2024 2024-04-23T21:55:14.512 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:55:14 2024 2024-04-23T21:55:14.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 26f94cd88cfe 2024-04-23T21:55:14.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:55:14.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:55:14.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: e1f65995-fd6c-4af2-8609-00268083e04f 2024-04-23T21:55:14.513 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: true 2024-04-23T21:55:14.514 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_6 2024-04-23T21:55:14.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:55:14.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:14.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:55:14.515 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:55:14.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=12 2024-04-23T21:55:14.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:55:14.517 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_6 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:55:15.023 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9572084672052178) > weight (0.25) 2024-04-23T21:55:15.023 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8231049350404076) > weight (0.25) 2024-04-23T21:55:15.023 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5220292363618649) > weight (0.25) 2024-04-23T21:55:15.023 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5610503563591069) > weight (0.25) 2024-04-23T21:55:15.024 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2024-04-23T21:55:19.521 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 70502400 bytes 2024-04-23T21:55:21.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:55:21.579+0000 7f9885caf640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x555f6518a840 finish: resending after timeout 2024-04-23T21:55:23.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 130488320 bytes 2024-04-23T21:55:25.257 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 163385344 bytes 2024-04-23T21:55:25.991 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 164002816 bytes 2024-04-23T21:55:26.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:55:26.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:55:26.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:55:26.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:55:26.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_6 present 2024-04-23T21:55:26.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:55:26.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:26.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:55:26.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=present 2024-04-23T21:55:26.528 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:55:26.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:55:26.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:55:26.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_6 2024-04-23T21:55:26.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:55:26.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:55:26.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_6 2024-04-23T21:55:26.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_6 2024-04-23T21:55:26.529 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:55:26.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id=121d3af59ab 2024-04-23T21:55:26.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:55:26.578 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:55:26.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_6 present 121d3af59ab 2024-04-23T21:55:26.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:55:26.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:26.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:55:26.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=present 2024-04-23T21:55:26.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id=121d3af59ab 2024-04-23T21:55:26.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:55:26.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:55:26.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_6 2024-04-23T21:55:26.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:55:26.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:55:26.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_6 2024-04-23T21:55:26.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_6 2024-04-23T21:55:26.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:55:26.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id=121d3af59ab 2024-04-23T21:55:26.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 121d3af59ab 2024-04-23T21:55:26.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -z 121d3af59ab -o 121d3af59ab = 121d3af59ab 2024-04-23T21:55:26.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=present 2024-04-23T21:55:26.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test present = present 2024-04-23T21:55:26.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:55:26.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ disable_mirror cluster2 mirror image_6 2024-04-23T21:55:26.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:55:26.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:26.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:55:26.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_6 2024-04-23T21:55:55.598 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring disabled 2024-04-23T21:55:55.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_6 deleted 2024-04-23T21:55:55.609 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:55:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:55:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:55:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:55:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:55:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:55:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_6 2024-04-23T21:55:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:55:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:55:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_6 2024-04-23T21:55:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_6 2024-04-23T21:55:55.610 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:55:55.642 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_6: (2) No such file or directory 2024-04-23T21:55:55.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:55:55.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:55:55.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:55:55.746 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_6 deleted '' 2024-04-23T21:55:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:55:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:55:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:55:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:55:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:55:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:55:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_6 2024-04-23T21:55:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:55:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:55:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_6 2024-04-23T21:55:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_6 2024-04-23T21:55:55.747 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:55:55.780 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_6: (2) No such file or directory 2024-04-23T21:55:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:55:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:55:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:55:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:55:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ purge_snapshots cluster2 mirror image_6 2024-04-23T21:55:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:55:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:55:55.784 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap purge mirror/image_6 2024-04-23T21:55:55.828 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_6 2024-04-23T21:55:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:55:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:55:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:55:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:55:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_6 2024-04-23T21:55:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:55:55.829 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:55.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_6 2024-04-23T21:55:55.830 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_6 2024-04-23T21:55:55.873 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_6 2024-04-23T21:55:56.098 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-04-23T21:55:56.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:55:56.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:55:56.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_7 2024-04-23T21:55:56.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_7 512M 2024-04-23T21:55:56.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:55:56.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:55:56.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:56.102 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:55:56.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:55:56.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:55:56.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:55:56.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 512M ']' 2024-04-23T21:55:56.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=512M 2024-04-23T21:55:56.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:55:56.103 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering,exclusive-lock,journaling mirror/image_7 2024-04-23T21:55:56.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ enable_mirror cluster2 mirror image_7 2024-04-23T21:55:56.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:55:56.362 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:56.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:55:56.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:55:56.363 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_7 journal 2024-04-23T21:55:56.407 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring enabled 2024-04-23T21:55:56.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 info mirror/image_7 2024-04-23T21:55:56.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_7': 2024-04-23T21:55:56.449 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:55:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:55:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 0 2024-04-23T21:55:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 270a781621b 2024-04-23T21:55:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.270a781621b 2024-04-23T21:55:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:55:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:55:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:55:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:55:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:55:56 2024 2024-04-23T21:55:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:55:56 2024 2024-04-23T21:55:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:55:56 2024 2024-04-23T21:55:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 270a781621b 2024-04-23T21:55:56.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:55:56.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:55:56.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 28885328-37da-4cc6-9411-7d47dce77066 2024-04-23T21:55:56.451 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: true 2024-04-23T21:55:56.452 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_7 2024-04-23T21:55:56.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:55:56.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:55:56.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:55:56.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:55:56.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=14 2024-04-23T21:55:56.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:55:56.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_7 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:55:59.128 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 43754496 bytes 2024-04-23T21:55:59.637 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 47590400 bytes 2024-04-23T21:56:01.730 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 79478784 bytes 2024-04-23T21:56:04.898 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 136328192 bytes 2024-04-23T21:56:07.781 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 173189120 bytes 2024-04-23T21:56:08.589 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:600 IOs, 202310656 bytes 2024-04-23T21:56:09.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:700 IOs, 202911744 bytes 2024-04-23T21:56:10.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:56:10.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:56:10.464 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:56:10.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:56:10.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_7 present 2024-04-23T21:56:10.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:56:10.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:56:10.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:56:10.465 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=present 2024-04-23T21:56:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:56:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:56:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:56:10.466 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_7 2024-04-23T21:56:10.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:56:10.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:56:10.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:56:10.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_7 2024-04-23T21:56:10.467 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:56:10.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id=121d55d7fdd3 2024-04-23T21:56:10.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:56:10.510 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:56:10.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_7 present 121d55d7fdd3 2024-04-23T21:56:10.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:56:10.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:56:10.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:56:10.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=present 2024-04-23T21:56:10.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id=121d55d7fdd3 2024-04-23T21:56:10.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:56:10.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:56:10.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_7 2024-04-23T21:56:10.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:56:10.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:56:10.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:56:10.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_7 2024-04-23T21:56:10.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:56:10.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id=121d55d7fdd3 2024-04-23T21:56:10.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 121d55d7fdd3 2024-04-23T21:56:10.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -z 121d55d7fdd3 -o 121d55d7fdd3 = 121d55d7fdd3 2024-04-23T21:56:10.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=present 2024-04-23T21:56:10.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test present = present 2024-04-23T21:56:10.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:56:10.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ disable_mirror cluster2 mirror image_7 2024-04-23T21:56:10.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:56:10.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:56:10.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:56:10.656 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_7 2024-04-23T21:56:22.426 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6985277717536681) > weight (0.25) 2024-04-23T21:56:22.427 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8724175876602914) > weight (0.25) 2024-04-23T21:56:22.427 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6691983759944875) > weight (0.25) 2024-04-23T21:56:22.427 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4280483080107481) > weight (0.25) 2024-04-23T21:56:22.427 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2024-04-23T21:56:40.857 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring disabled 2024-04-23T21:56:40.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_7 deleted 2024-04-23T21:56:40.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:56:40.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:56:40.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:56:40.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:56:40.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:56:40.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:56:40.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:56:40.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_7 2024-04-23T21:56:40.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:56:40.865 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:56:40.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:56:40.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_7 2024-04-23T21:56:40.866 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:56:40.897 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_7: (2) No such file or directory 2024-04-23T21:56:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:56:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:56:40.900 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:56:41.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_7 deleted '' 2024-04-23T21:56:41.001 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:56:41.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:56:41.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:56:41.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:56:41.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:56:41.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:56:41.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:56:41.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_7 2024-04-23T21:56:41.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:56:41.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:56:41.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_7 2024-04-23T21:56:41.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_7 2024-04-23T21:56:41.002 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:56:41.035 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_7: (2) No such file or directory 2024-04-23T21:56:41.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:56:41.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:56:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:56:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:56:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ purge_snapshots cluster2 mirror image_7 2024-04-23T21:56:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:56:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:56:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:56:41.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap purge mirror/image_7 2024-04-23T21:56:41.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_7 2024-04-23T21:56:41.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:56:41.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:56:41.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:56:41.082 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:56:41.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:56:41.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_7 2024-04-23T21:56:41.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:56:41.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:56:41.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_7 2024-04-23T21:56:41.083 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_7 2024-04-23T21:56:41.127 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_7 2024-04-23T21:56:41.341 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-04-23T21:56:41.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:56:41.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:56:41.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_8 2024-04-23T21:56:41.344 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_8 512M 2024-04-23T21:56:41.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:56:41.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:56:41.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:56:41.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:56:41.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:56:41.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:56:41.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:56:41.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 512M ']' 2024-04-23T21:56:41.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=512M 2024-04-23T21:56:41.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:56:41.345 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering,exclusive-lock,journaling mirror/image_8 2024-04-23T21:56:41.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ enable_mirror cluster2 mirror image_8 2024-04-23T21:56:41.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:56:41.399 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:56:41.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:56:41.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:56:41.400 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_8 journal 2024-04-23T21:56:41.445 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring enabled 2024-04-23T21:56:41.450 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 info mirror/image_8 2024-04-23T21:56:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_8': 2024-04-23T21:56:41.486 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:56:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:56:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 0 2024-04-23T21:56:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 271c6ac95280 2024-04-23T21:56:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.271c6ac95280 2024-04-23T21:56:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:56:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:56:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:56:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:56:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:56:41 2024 2024-04-23T21:56:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:56:41 2024 2024-04-23T21:56:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:56:41 2024 2024-04-23T21:56:41.487 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 271c6ac95280 2024-04-23T21:56:41.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:56:41.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:56:41.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 1f1487d8-e43f-4497-9741-d50479dc43b0 2024-04-23T21:56:41.488 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: true 2024-04-23T21:56:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_8 2024-04-23T21:56:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:56:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:56:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:56:41.490 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:56:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=12 2024-04-23T21:56:41.491 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:56:41.492 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_8 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:56:44.579 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 51974144 bytes 2024-04-23T21:56:47.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:56:47.751+0000 7f99a2eb5640 -1 rbd::mirror::image_replayer::journal::Replayer: 0x5619adfe1c00 get_replay_status: replay not running 2024-04-23T21:56:47.863 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 109474816 bytes 2024-04-23T21:56:49.373 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 134123520 bytes 2024-04-23T21:56:50.424 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 148055040 bytes 2024-04-23T21:56:51.276 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 166346752 bytes 2024-04-23T21:56:53.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:56:53.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:56:53.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:56:53.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:56:53.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_8 present 2024-04-23T21:56:53.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:56:53.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:56:53.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:56:53.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=present 2024-04-23T21:56:53.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:56:53.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:56:53.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:56:53.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_8 2024-04-23T21:56:53.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:56:53.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:56:53.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:56:53.502 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_8 2024-04-23T21:56:53.503 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:56:53.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id=121dac719dc4 2024-04-23T21:56:53.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:56:53.546 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:56:53.647 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_8 present 121dac719dc4 2024-04-23T21:56:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:56:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:56:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:56:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=present 2024-04-23T21:56:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id=121dac719dc4 2024-04-23T21:56:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:56:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:56:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_8 2024-04-23T21:56:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:56:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:56:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:56:53.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_8 2024-04-23T21:56:53.649 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:56:53.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id=121dac719dc4 2024-04-23T21:56:53.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 121dac719dc4 2024-04-23T21:56:53.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -z 121dac719dc4 -o 121dac719dc4 = 121dac719dc4 2024-04-23T21:56:53.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=present 2024-04-23T21:56:53.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test present = present 2024-04-23T21:56:53.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:56:53.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ disable_mirror cluster2 mirror image_8 2024-04-23T21:56:53.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:56:53.692 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:56:53.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:56:53.693 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_8 2024-04-23T21:57:09.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6650510324900311) > weight (0.25) 2024-04-23T21:57:09.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5314997238273781) > weight (0.25) 2024-04-23T21:57:09.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.34952035192448194) > weight (0.25) 2024-04-23T21:57:09.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.350459196767139) > weight (0.25) 2024-04-23T21:57:09.024 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2024-04-23T21:57:22.608 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring disabled 2024-04-23T21:57:22.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_8 deleted 2024-04-23T21:57:22.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:57:22.615 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:57:22.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:57:22.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:57:22.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:57:22.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:57:22.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:57:22.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_8 2024-04-23T21:57:22.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:57:22.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:57:22.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:57:22.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_8 2024-04-23T21:57:22.616 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:57:22.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_8: (2) No such file or directory 2024-04-23T21:57:22.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:57:22.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:57:22.651 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:57:22.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_8 deleted '' 2024-04-23T21:57:22.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:57:22.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:57:22.753 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:57:22.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:57:22.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:57:22.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:57:22.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:57:22.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_8 2024-04-23T21:57:22.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:57:22.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:57:22.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_8 2024-04-23T21:57:22.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_8 2024-04-23T21:57:22.754 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:57:22.785 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_8: (2) No such file or directory 2024-04-23T21:57:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:57:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:57:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:57:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:57:22.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ purge_snapshots cluster2 mirror image_8 2024-04-23T21:57:22.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:57:22.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:57:22.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:57:22.789 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap purge mirror/image_8 2024-04-23T21:57:22.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_8 2024-04-23T21:57:22.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:57:22.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:57:22.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:57:22.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:57:22.831 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:57:22.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_8 2024-04-23T21:57:22.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:57:22.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:57:22.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_8 2024-04-23T21:57:22.832 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_8 2024-04-23T21:57:22.876 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_8 2024-04-23T21:57:23.100 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-04-23T21:57:23.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:57:23.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:57:23.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_9 2024-04-23T21:57:23.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_9 512M 2024-04-23T21:57:23.104 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:57:23.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:57:23.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:57:23.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:57:23.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:57:23.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:57:23.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:57:23.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 512M ']' 2024-04-23T21:57:23.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=512M 2024-04-23T21:57:23.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:57:23.105 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering,exclusive-lock,journaling mirror/image_9 2024-04-23T21:57:23.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ enable_mirror cluster2 mirror image_9 2024-04-23T21:57:23.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:57:23.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:57:23.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:57:23.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:57:23.160 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_9 journal 2024-04-23T21:57:23.204 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring enabled 2024-04-23T21:57:23.210 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 info mirror/image_9 2024-04-23T21:57:23.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_9': 2024-04-23T21:57:23.246 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:57:23.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:57:23.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 0 2024-04-23T21:57:23.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 272eee96fc6f 2024-04-23T21:57:23.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.272eee96fc6f 2024-04-23T21:57:23.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:57:23.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:57:23.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:57:23.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:57:23.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:57:23 2024 2024-04-23T21:57:23.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:57:23 2024 2024-04-23T21:57:23.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:57:23 2024 2024-04-23T21:57:23.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 272eee96fc6f 2024-04-23T21:57:23.247 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:57:23.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:57:23.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 5eb069a9-6d2d-4680-9b67-277fb29cf8b5 2024-04-23T21:57:23.248 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: true 2024-04-23T21:57:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_9 2024-04-23T21:57:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:57:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:57:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:57:23.250 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:57:23.251 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=14 2024-04-23T21:57:23.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:57:23.252 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_9 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:57:27.701 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 52386816 bytes 2024-04-23T21:57:31.542 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 128433152 bytes 2024-04-23T21:57:33.176 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:300 IOs, 165839872 bytes 2024-04-23T21:57:35.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:400 IOs, 192708608 bytes 2024-04-23T21:57:37.059 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:500 IOs, 222072832 bytes 2024-04-23T21:57:37.263 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:57:37.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:57:37.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:57:37.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:57:37.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_9 present 2024-04-23T21:57:37.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:57:37.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:57:37.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:57:37.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=present 2024-04-23T21:57:37.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:57:37.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:57:37.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:57:37.264 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_9 2024-04-23T21:57:37.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:57:37.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:57:37.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:57:37.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_9 2024-04-23T21:57:37.265 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:57:37.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id=121dc24905ce 2024-04-23T21:57:37.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:57:37.310 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:57:37.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_9 present 121dc24905ce 2024-04-23T21:57:37.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:57:37.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:57:37.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:57:37.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=present 2024-04-23T21:57:37.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id=121dc24905ce 2024-04-23T21:57:37.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:57:37.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:57:37.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_9 2024-04-23T21:57:37.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:57:37.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:57:37.412 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:57:37.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_9 2024-04-23T21:57:37.413 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:57:37.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id=121dc24905ce 2024-04-23T21:57:37.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 121dc24905ce 2024-04-23T21:57:37.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -z 121dc24905ce -o 121dc24905ce = 121dc24905ce 2024-04-23T21:57:37.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=present 2024-04-23T21:57:37.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test present = present 2024-04-23T21:57:37.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:57:37.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ disable_mirror cluster2 mirror image_9 2024-04-23T21:57:37.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:57:37.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:57:37.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:57:37.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_9 2024-04-23T21:58:04.428 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3099985469993224) > weight (0.25) 2024-04-23T21:58:04.429 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.36068604550524375) > weight (0.25) 2024-04-23T21:58:04.429 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2024-04-23T21:58:04.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2024-04-23T21:58:04.429 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2024-04-23T21:58:04.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:04.430+0000 7fa4213dc640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 104895) UID: 1000 2024-04-23T21:58:07.816 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring disabled 2024-04-23T21:58:07.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_9 deleted 2024-04-23T21:58:07.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:58:07.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:07.823 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:58:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:58:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:58:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:58:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:58:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_9 2024-04-23T21:58:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:58:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:58:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_9 2024-04-23T21:58:07.824 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:58:07.856 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_9: (2) No such file or directory 2024-04-23T21:58:07.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id= 2024-04-23T21:58:07.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:58:07.859 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:58:07.960 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_9 deleted '' 2024-04-23T21:58:07.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:58:07.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:07.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:58:07.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:58:07.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:58:07.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:58:07.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:58:07.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_9 2024-04-23T21:58:07.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:58:07.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:07.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_9 2024-04-23T21:58:07.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_9 2024-04-23T21:58:07.961 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:58:07.993 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_9: (2) No such file or directory 2024-04-23T21:58:07.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:58:07.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:58:07.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:58:07.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:58:07.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ purge_snapshots cluster2 mirror image_9 2024-04-23T21:58:07.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:58:07.996 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:07.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:58:07.997 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap purge mirror/image_9 2024-04-23T21:58:08.037 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_9 2024-04-23T21:58:08.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:58:08.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:08.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:58:08.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:58:08.038 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:58:08.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_9 2024-04-23T21:58:08.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:58:08.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:08.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_9 2024-04-23T21:58:08.039 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_9 2024-04-23T21:58:08.081 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_9 2024-04-23T21:58:08.294 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-04-23T21:58:08.297 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:58:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for i in `seq 1 10` 2024-04-23T21:58:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image=image_10 2024-04-23T21:58:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ create_image cluster2 mirror image_10 512M 2024-04-23T21:58:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:58:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:58:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:58:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:58:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:58:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local size=128 2024-04-23T21:58:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -n 512M ']' 2024-04-23T21:58:08.298 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ size=512M 2024-04-23T21:58:08.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ shift 2024-04-23T21:58:08.299 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering,exclusive-lock,journaling mirror/image_10 2024-04-23T21:58:08.351 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ enable_mirror cluster2 mirror image_10 2024-04-23T21:58:08.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:58:08.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:08.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:58:08.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local mode=journal 2024-04-23T21:58:08.353 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_10 journal 2024-04-23T21:58:08.397 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring enabled 2024-04-23T21:58:08.402 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 info mirror/image_10 2024-04-23T21:58:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:rbd image 'image_10': 2024-04-23T21:58:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: size 512 MiB in 128 objects 2024-04-23T21:58:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: order 22 (4 MiB objects) 2024-04-23T21:58:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: snapshot_count: 0 2024-04-23T21:58:08.439 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: id: 27422795ae9f 2024-04-23T21:58:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: block_name_prefix: rbd_data.27422795ae9f 2024-04-23T21:58:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: format: 2 2024-04-23T21:58:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: features: layering, exclusive-lock, journaling 2024-04-23T21:58:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: op_features: 2024-04-23T21:58:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: flags: 2024-04-23T21:58:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: create_timestamp: Tue Apr 23 21:58:08 2024 2024-04-23T21:58:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: access_timestamp: Tue Apr 23 21:58:08 2024 2024-04-23T21:58:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: modify_timestamp: Tue Apr 23 21:58:08 2024 2024-04-23T21:58:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: journal: 27422795ae9f 2024-04-23T21:58:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring state: enabled 2024-04-23T21:58:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring mode: journal 2024-04-23T21:58:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring global id: 41dad2f0-937f-406d-9ebe-49d1948e0f05 2024-04-23T21:58:08.440 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout: mirroring primary: true 2024-04-23T21:58:08.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stress_write_image cluster2 mirror image_10 2024-04-23T21:58:08.442 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:58:08.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:08.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:58:08.443 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2024-04-23T21:58:08.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local duration=6 2024-04-23T21:58:08.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:58:08.444 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_10 --debug-rbd=20 --debug-journaler=20 2024-04-23T21:58:09.915 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:100 IOs, 23575552 bytes 2024-04-23T21:58:11.430 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2024-04-23T21:58:11.430 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:58:11.431 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2024-04-23T21:58:11.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2024-04-23T21:58:11.431 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2024-04-23T21:58:11.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2024-04-23T21:58:11.434 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2024-04-23T21:58:12.548 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:200 IOs, 78156800 bytes 2024-04-23T21:58:14.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=124 2024-04-23T21:58:14.454 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:58:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 124 -eq 124 ']' 2024-04-23T21:58:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:58:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_10 present 2024-04-23T21:58:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:58:14.455 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:14.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:58:14.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=present 2024-04-23T21:58:14.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:58:14.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:58:14.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:58:14.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_10 2024-04-23T21:58:14.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:58:14.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:14.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:58:14.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_10 2024-04-23T21:58:14.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:58:14.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id=121d505f44b2 2024-04-23T21:58:14.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:58:14.525 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:58:14.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_10 present 121d505f44b2 2024-04-23T21:58:14.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:58:14.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:14.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:58:14.627 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=present 2024-04-23T21:58:14.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id=121d505f44b2 2024-04-23T21:58:14.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:58:14.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:58:14.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_10 2024-04-23T21:58:14.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:58:14.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:14.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:58:14.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_10 2024-04-23T21:58:14.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:58:14.673 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id=121d505f44b2 2024-04-23T21:58:14.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 121d505f44b2 2024-04-23T21:58:14.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -z 121d505f44b2 -o 121d505f44b2 = 121d505f44b2 2024-04-23T21:58:14.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_state=present 2024-04-23T21:58:14.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test present = present 2024-04-23T21:58:14.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:58:14.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ disable_mirror cluster2 mirror image_10 2024-04-23T21:58:14.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:58:14.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:14.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:58:14.674 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_10 2024-04-23T21:58:16.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:16.517+0000 7f98854ae640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x555f65640840 finish: resending after timeout 2024-04-23T21:58:21.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:21.518+0000 7f9885caf640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x555f65640840 finish: resending after timeout 2024-04-23T21:58:26.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:26.519+0000 7f98854ae640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x555f65640840 finish: resending after timeout 2024-04-23T21:58:31.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:31.520+0000 7f9885caf640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x555f65640840 finish: resending after timeout 2024-04-23T21:58:36.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:36.521+0000 7f9885caf640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x555f65640840 finish: resending after timeout 2024-04-23T21:58:41.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:41.521+0000 7f98854ae640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x555f65640840 finish: resending after timeout 2024-04-23T21:58:46.025 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5775018600323707) > weight (0.25) 2024-04-23T21:58:46.026 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8229148032378726) > weight (0.25) 2024-04-23T21:58:46.026 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2024-04-23T21:58:46.026 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2024-04-23T21:58:46.026 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2024-04-23T21:58:46.027 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi150.stderr:2024-04-23T21:58:46.026+0000 7fe32c649640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 83498) UID: 1000 2024-04-23T21:58:46.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:46.522+0000 7f9885caf640 -1 rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x555f65640840 finish: resending after timeout 2024-04-23T21:58:48.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:48.885+0000 7f999669c640 -1 rbd::mirror::image_replayer::journal::SyncPointHandler: 0x5619a9a58510 handle_update_sync_points: failed to update remote journal client meta for image 41dad2f0-937f-406d-9ebe-49d1948e0f05: (2) No such file or directory 2024-04-23T21:58:48.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:48.885+0000 7f999669c640 -1 rbd::mirror::image_sync::SyncPointPruneRequest: 0x5619adf96cc0 handle_update_sync_points: failed to update client data: (2) No such file or directory 2024-04-23T21:58:48.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:48.885+0000 7f999669c640 -1 rbd::mirror::ImageSync: 0x5619ae77b080 handle_prune_sync_points: failed to prune sync point: (2) No such file or directory 2024-04-23T21:58:48.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:48.885+0000 7f999669c640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x5619a91aba00 handle_image_sync: failed to sync remote image: (2) No such file or directory 2024-04-23T21:58:48.948 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:Mirroring disabled 2024-04-23T21:58:48.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_present cluster1 mirror image_10 deleted 2024-04-23T21:58:48.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:58:48.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:48.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:58:48.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local state=deleted 2024-04-23T21:58:48.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id= 2024-04-23T21:58:48.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local s 2024-04-23T21:58:48.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:58:48.955 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_10 2024-04-23T21:58:48.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:58:48.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:48.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:58:48.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_10 2024-04-23T21:58:48.956 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:58:49.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ image_id=121d505f44b2 2024-04-23T21:58:49.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2024-04-23T21:58:49.005 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0.1 2024-04-23T21:58:49.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test_image_present cluster1 mirror image_10 deleted 121d505f44b2 2024-04-23T21:58:49.107 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:58:49.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:49.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:58:49.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local test_state=deleted 2024-04-23T21:58:49.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image_id=121d505f44b2 2024-04-23T21:58:49.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_state=deleted 2024-04-23T21:58:49.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local current_image_id 2024-04-23T21:58:49.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ get_image_id cluster1 mirror image_10 2024-04-23T21:58:49.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:58:49.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:49.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local image=image_10 2024-04-23T21:58:49.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rbd --cluster=cluster1 info mirror/image_10 2024-04-23T21:58:49.108 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2024-04-23T21:58:49.139 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:rbd: error opening image image_10: (2) No such file or directory 2024-04-23T21:58:49.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ current_image_id= 2024-04-23T21:58:49.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n '' 2024-04-23T21:58:49.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test deleted = deleted 2024-04-23T21:58:49.142 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:58:49.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ purge_snapshots cluster2 mirror image_10 2024-04-23T21:58:49.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:58:49.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:49.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:58:49.143 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster cluster2 snap purge mirror/image_10 2024-04-23T21:58:49.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image_retry cluster2 mirror image_10 2024-04-23T21:58:49.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:58:49.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:49.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:58:49.185 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 4 8 16 32 2024-04-23T21:58:49.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:58:49.186 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ remove_image cluster2 mirror image_10 2024-04-23T21:58:49.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:58:49.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:49.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local image=image_10 2024-04-23T21:58:49.187 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_10 2024-04-23T21:58:49.229 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rbd --cluster=cluster2 rm mirror/image_10 2024-04-23T21:58:49.453 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2024-04-23T21:58:49.456 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:58:49.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ testlog 'TEST: check if removed images'\'' OMAP are removed' 2024-04-23T21:58:49.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ tee -a /tmp/tmp.8Tyc9aSFBO/rbd-mirror.test.log 2024-04-23T21:58:49.457 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ date '+%F %T' 2024-04-23T21:58:49.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo 2024-04-23 21:58:49 TEST: check if removed 'images'\''' OMAP are removed 2024-04-23T21:58:49.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:2024-04-23 21:58:49 TEST: check if removed images' OMAP are removed 2024-04-23T21:58:49.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_in_omap cluster1 mirror 2024-04-23T21:58:49.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:58:49.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:49.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_omap_keys cluster1 mirror rbd_mirroring status_global 2024-04-23T21:58:49.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:58:49.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:49.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local obj_name=rbd_mirroring 2024-04-23T21:58:49.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local filter=status_global 2024-04-23T21:58:49.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 2 4 4 8 8 8 16 16 32 2024-04-23T21:58:49.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:58:49.459 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:58:49.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ count_omap_keys_with_filter cluster1 mirror rbd_mirroring status_global 2024-04-23T21:58:49.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:58:49.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:49.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local obj_name=rbd_mirroring 2024-04-23T21:58:49.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local filter=status_global 2024-04-23T21:58:49.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ list_omap_keys cluster1 mirror rbd_mirroring 2024-04-23T21:58:49.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:58:49.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:49.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local obj_name=rbd_mirroring 2024-04-23T21:58:49.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -c status_global 2024-04-23T21:58:49.461 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rados --cluster cluster1 -p mirror listomapkeys rbd_mirroring 2024-04-23T21:58:49.611 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 0 = 0 2024-04-23T21:58:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=0 2024-04-23T21:58:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:58:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 -eq 0 ']' 2024-04-23T21:58:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:58:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_omap_keys cluster1 mirror rbd_mirroring image_ 2024-04-23T21:58:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:58:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local obj_name=rbd_mirroring 2024-04-23T21:58:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local filter=image_ 2024-04-23T21:58:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 2 4 4 8 8 8 16 16 32 2024-04-23T21:58:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:58:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:58:49.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ count_omap_keys_with_filter cluster1 mirror rbd_mirroring image_ 2024-04-23T21:58:49.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:58:49.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:49.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local obj_name=rbd_mirroring 2024-04-23T21:58:49.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local filter=image_ 2024-04-23T21:58:49.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ list_omap_keys cluster1 mirror rbd_mirroring 2024-04-23T21:58:49.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:58:49.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:49.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local obj_name=rbd_mirroring 2024-04-23T21:58:49.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -c image_ 2024-04-23T21:58:49.613 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rados --cluster cluster1 -p mirror listomapkeys rbd_mirroring 2024-04-23T21:58:49.645 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 0 = 0 2024-04-23T21:58:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=0 2024-04-23T21:58:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:58:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 -eq 0 ']' 2024-04-23T21:58:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:58:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_omap_keys cluster1 mirror rbd_mirror_leader image_map 2024-04-23T21:58:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:58:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local obj_name=rbd_mirror_leader 2024-04-23T21:58:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local filter=image_map 2024-04-23T21:58:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 2 4 4 8 8 8 16 16 32 2024-04-23T21:58:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:58:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:58:49.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ count_omap_keys_with_filter cluster1 mirror rbd_mirror_leader image_map 2024-04-23T21:58:49.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:58:49.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:49.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local obj_name=rbd_mirror_leader 2024-04-23T21:58:49.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local filter=image_map 2024-04-23T21:58:49.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ list_omap_keys cluster1 mirror rbd_mirror_leader 2024-04-23T21:58:49.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:58:49.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:49.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local obj_name=rbd_mirror_leader 2024-04-23T21:58:49.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -c image_map 2024-04-23T21:58:49.648 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rados --cluster cluster1 -p mirror listomapkeys rbd_mirror_leader 2024-04-23T21:58:49.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 1 = 0 2024-04-23T21:58:49.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=1 2024-04-23T21:58:49.679 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:58:49.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 -eq 0 ']' 2024-04-23T21:58:49.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 2 4 4 8 8 8 16 16 32 2024-04-23T21:58:49.680 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 1 2024-04-23T21:58:50.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:58:50.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ count_omap_keys_with_filter cluster1 mirror rbd_mirror_leader image_map 2024-04-23T21:58:50.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:58:50.681 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local obj_name=rbd_mirror_leader 2024-04-23T21:58:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local filter=image_map 2024-04-23T21:58:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ list_omap_keys cluster1 mirror rbd_mirror_leader 2024-04-23T21:58:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:58:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -c image_map 2024-04-23T21:58:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local obj_name=rbd_mirror_leader 2024-04-23T21:58:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rados --cluster cluster1 -p mirror listomapkeys rbd_mirror_leader 2024-04-23T21:58:50.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 1 = 0 2024-04-23T21:58:50.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=1 2024-04-23T21:58:50.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:58:50.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 1 -eq 0 ']' 2024-04-23T21:58:50.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 2 4 4 8 8 8 16 16 32 2024-04-23T21:58:50.720 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 2 2024-04-23T21:58:52.721 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:58:52.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ count_omap_keys_with_filter cluster1 mirror rbd_mirror_leader image_map 2024-04-23T21:58:52.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:58:52.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:52.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local obj_name=rbd_mirror_leader 2024-04-23T21:58:52.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local filter=image_map 2024-04-23T21:58:52.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ list_omap_keys cluster1 mirror rbd_mirror_leader 2024-04-23T21:58:52.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster1 2024-04-23T21:58:52.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:52.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local obj_name=rbd_mirror_leader 2024-04-23T21:58:52.722 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -c image_map 2024-04-23T21:58:52.723 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rados --cluster cluster1 -p mirror listomapkeys rbd_mirror_leader 2024-04-23T21:58:52.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 0 = 0 2024-04-23T21:58:52.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=0 2024-04-23T21:58:52.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:58:52.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 -eq 0 ']' 2024-04-23T21:58:52.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:58:52.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_image_in_omap cluster2 mirror 2024-04-23T21:58:52.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:58:52.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:52.761 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_omap_keys cluster2 mirror rbd_mirroring status_global 2024-04-23T21:58:52.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:58:52.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:52.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local obj_name=rbd_mirroring 2024-04-23T21:58:52.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local filter=status_global 2024-04-23T21:58:52.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 2 4 4 8 8 8 16 16 32 2024-04-23T21:58:52.762 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:58:52.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:58:52.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ count_omap_keys_with_filter cluster2 mirror rbd_mirroring status_global 2024-04-23T21:58:52.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:58:52.763 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:52.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local obj_name=rbd_mirroring 2024-04-23T21:58:52.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local filter=status_global 2024-04-23T21:58:52.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ list_omap_keys cluster2 mirror rbd_mirroring 2024-04-23T21:58:52.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:58:52.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:52.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local obj_name=rbd_mirroring 2024-04-23T21:58:52.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -c status_global 2024-04-23T21:58:52.764 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rados --cluster cluster2 -p mirror listomapkeys rbd_mirroring 2024-04-23T21:58:52.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 0 = 0 2024-04-23T21:58:52.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=0 2024-04-23T21:58:52.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:58:52.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 -eq 0 ']' 2024-04-23T21:58:52.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:58:52.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_omap_keys cluster2 mirror rbd_mirroring image_ 2024-04-23T21:58:52.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:58:52.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:52.800 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local obj_name=rbd_mirroring 2024-04-23T21:58:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local filter=image_ 2024-04-23T21:58:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 2 4 4 8 8 8 16 16 32 2024-04-23T21:58:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:58:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:58:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ count_omap_keys_with_filter cluster2 mirror rbd_mirroring image_ 2024-04-23T21:58:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:58:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local obj_name=rbd_mirroring 2024-04-23T21:58:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local filter=image_ 2024-04-23T21:58:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ list_omap_keys cluster2 mirror rbd_mirroring 2024-04-23T21:58:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:58:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local obj_name=rbd_mirroring 2024-04-23T21:58:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rados --cluster cluster2 -p mirror listomapkeys rbd_mirroring 2024-04-23T21:58:52.801 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -c image_ 2024-04-23T21:58:52.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 0 = 0 2024-04-23T21:58:52.834 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=0 2024-04-23T21:58:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:58:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 -eq 0 ']' 2024-04-23T21:58:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:58:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ wait_for_omap_keys cluster2 mirror rbd_mirror_leader image_map 2024-04-23T21:58:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:58:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local pool=mirror 2024-04-23T21:58:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local obj_name=rbd_mirror_leader 2024-04-23T21:58:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local filter=image_map 2024-04-23T21:58:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for s in 0 1 2 2 4 4 8 8 8 16 16 32 2024-04-23T21:58:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ sleep 0 2024-04-23T21:58:52.835 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:58:52.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ count_omap_keys_with_filter cluster2 mirror rbd_mirror_leader image_map 2024-04-23T21:58:52.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:58:52.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:52.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local obj_name=rbd_mirror_leader 2024-04-23T21:58:52.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local filter=image_map 2024-04-23T21:58:52.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ list_omap_keys cluster2 mirror rbd_mirror_leader 2024-04-23T21:58:52.837 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local cluster=cluster2 2024-04-23T21:58:52.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local pool=mirror 2024-04-23T21:58:52.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ local obj_name=rbd_mirror_leader 2024-04-23T21:58:52.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ grep -c image_map 2024-04-23T21:58:52.838 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ rados --cluster cluster2 -p mirror listomapkeys rbd_mirror_leader 2024-04-23T21:58:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test 0 = 0 2024-04-23T21:58:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ error_code=0 2024-04-23T21:58:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set -e 2024-04-23T21:58:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 -eq 0 ']' 2024-04-23T21:58:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 0 2024-04-23T21:58:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ cleanup 0 2024-04-23T21:58:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local error_code=0 2024-04-23T21:58:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ set +e 2024-04-23T21:58:52.869 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 -ne 0 ']' 2024-04-23T21:58:52.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -z '' ']' 2024-04-23T21:58:52.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster instance 2024-04-23T21:58:52.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ CEPH_ARGS= 2024-04-23T21:58:52.870 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --cluster cluster1 osd pool rm mirror mirror --yes-i-really-really-mean-it 2024-04-23T21:58:53.380 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:pool 'mirror' does not exist 2024-04-23T21:58:53.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ CEPH_ARGS= 2024-04-23T21:58:53.391 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --cluster cluster2 osd pool rm mirror mirror --yes-i-really-really-mean-it 2024-04-23T21:58:53.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:53.599+0000 7f9885caf640 -1 rbd::mirror::LeaderWatcher: 0x555f64aecc00 handle_notify_heartbeat: error notifying heartbeat: (2) No such file or directory, releasing leader 2024-04-23T21:58:53.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:53.600+0000 7f987f4a2640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x555f64af0fc0 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2024-04-23T21:58:53.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:53.600+0000 7f987eca1640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x555f65220000 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2024-04-23T21:58:53.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:53.600+0000 7f987f4a2640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x555f65101200 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2024-04-23T21:58:53.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:53.600+0000 7f987eca1640 -1 rbd::mirror::PoolWatcher: 0x555f65b70960 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-04-23T21:58:53.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:53.600+0000 7f987f4a2640 -1 rbd::mirror::PoolWatcher: 0x555f65b712c0 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-04-23T21:58:53.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:53.600+0000 7f987eca1640 -1 rbd::mirror::PoolWatcher: 0x555f65b71e00 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-04-23T21:58:53.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:53.606+0000 7f9885caf640 -1 rbd::mirror::LeaderWatcher: 0x555f64aecc00 handle_notify_lock_released: error notifying leader lock released: (2) No such file or directory 2024-04-23T21:58:53.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:53.606+0000 7f9885caf640 -1 rbd::mirror::LeaderWatcher: 0x555f64aecc00 handle_release_leader_lock: error releasing lock: (2) No such file or directory 2024-04-23T21:58:54.690 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:pool 'mirror' does not exist 2024-04-23T21:58:54.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ CEPH_ARGS= 2024-04-23T21:58:54.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --cluster cluster1 osd pool rm mirror_parent mirror_parent --yes-i-really-really-mean-it 2024-04-23T21:58:55.028 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2024-04-23T21:58:55.028 INFO:teuthology.orchestra.run:waiting for 600 2024-04-23T21:58:55.028 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2024-04-23T21:58:55.028 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2024-04-23T21:58:55.028 DEBUG:teuthology.orchestra.run.smithi150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2024-04-23T21:58:55.031 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2024-04-23T21:58:55.031 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2024-04-23T21:58:55.482 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:pool 'mirror_parent' does not exist 2024-04-23T21:58:55.500 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ CEPH_ARGS= 2024-04-23T21:58:55.501 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ ceph --cluster cluster2 osd pool rm mirror_parent mirror_parent --yes-i-really-really-mean-it 2024-04-23T21:58:55.689 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi150.stderr:2024-04-23T21:58:55.688+0000 7f95e441d640 -1 rbd::mirror::MirrorStatusWatcher: 0x560ed30446e0 operator(): error removing down statuses: (2) No such file or directory 2024-04-23T21:58:55.689 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi150.stderr:2024-04-23T21:58:55.688+0000 7f95e441d640 -1 rbd::mirror::MirrorStatusUpdater 0x560ed3fa26c0 handle_init_mirror_status_watcher: failed to init mirror status watcher: (2) No such file or directory 2024-04-23T21:58:55.689 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi150.stderr:2024-04-23T21:58:55.688+0000 7f95f1437640 -1 rbd::mirror::NamespaceReplayer: 0x560ed49ec000 handle_init_remote_status_updater: error initializing remote mirror status updater: (2) No such file or directory 2024-04-23T21:58:55.690 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi150.stderr:2024-04-23T21:58:55.689+0000 7f95f4c8e640 -1 rbd::mirror::PoolReplayer: 0x560ed2fd1b00 init: error initializing default namespace replayer: (2) No such file or directory 2024-04-23T21:58:56.116 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:56.115+0000 7f3e5be36640 -1 rbd::mirror::LeaderWatcher: 0x560e1a118c00 handle_notify_heartbeat: error notifying heartbeat: (2) No such file or directory, releasing leader 2024-04-23T21:58:56.116 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:56.115+0000 7f3e55e2a640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x560e1ab70fc0 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2024-04-23T21:58:56.116 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:56.115+0000 7f3e55e2a640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x560e1ab71d40 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2024-04-23T21:58:56.116 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:56.115+0000 7f3e55e2a640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x560e1a6e1d40 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2024-04-23T21:58:56.116 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:56.115+0000 7f3e55629640 -1 rbd::mirror::PoolWatcher: 0x560e1a113860 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-04-23T21:58:56.116 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:56.115+0000 7f3e55629640 -1 rbd::mirror::PoolWatcher: 0x560e1a7921e0 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-04-23T21:58:56.116 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:56.115+0000 7f3e55629640 -1 rbd::mirror::PoolWatcher: 0x560e1a6a01e0 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-04-23T21:58:56.116 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:56.115+0000 7f3e5c637640 -1 rbd::mirror::LeaderWatcher: 0x560e1a118c00 handle_notify_lock_released: error notifying leader lock released: (2) No such file or directory 2024-04-23T21:58:56.117 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:56.115+0000 7f3e5c637640 -1 rbd::mirror::LeaderWatcher: 0x560e1a118c00 handle_release_leader_lock: error releasing lock: (2) No such file or directory 2024-04-23T21:58:56.612 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:pool 'mirror_parent' does not exist 2024-04-23T21:58:56.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for cluster in "${CLUSTER1}" "${CLUSTER2}" 2024-04-23T21:58:56.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stop_mirrors cluster1 2024-04-23T21:58:56.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1 2024-04-23T21:58:56.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local sig= 2024-04-23T21:58:56.624 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:58:56.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:58:56.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stop_mirror cluster1:0 '' 2024-04-23T21:58:56.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:0 2024-04-23T21:58:56.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local sig= 2024-04-23T21:58:56.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:58:56.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:58:56.628 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:58:56.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stop_mirror cluster1:1 '' 2024-04-23T21:58:56.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:1 2024-04-23T21:58:56.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local sig= 2024-04-23T21:58:56.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:58:56.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:58:56.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:58:56.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stop_mirror cluster1:2 '' 2024-04-23T21:58:56.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:2 2024-04-23T21:58:56.629 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local sig= 2024-04-23T21:58:56.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:58:56.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:58:56.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:58:56.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stop_mirror cluster1:3 '' 2024-04-23T21:58:56.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster1:3 2024-04-23T21:58:56.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local sig= 2024-04-23T21:58:56.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:58:56.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:58:56.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for cluster in "${CLUSTER1}" "${CLUSTER2}" 2024-04-23T21:58:56.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stop_mirrors cluster2 2024-04-23T21:58:56.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2 2024-04-23T21:58:56.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local sig= 2024-04-23T21:58:56.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:++ seq 0 3 2024-04-23T21:58:56.630 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:58:56.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stop_mirror cluster2:0 '' 2024-04-23T21:58:56.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2:0 2024-04-23T21:58:56.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local sig= 2024-04-23T21:58:56.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:58:56.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:58:56.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:58:56.631 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stop_mirror cluster2:1 '' 2024-04-23T21:58:56.632 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2:1 2024-04-23T21:58:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local sig= 2024-04-23T21:58:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:58:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:58:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:58:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stop_mirror cluster2:2 '' 2024-04-23T21:58:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2:2 2024-04-23T21:58:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local sig= 2024-04-23T21:58:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:58:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:58:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2024-04-23T21:58:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ stop_mirror cluster2:3 '' 2024-04-23T21:58:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local cluster=cluster2:3 2024-04-23T21:58:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ local sig= 2024-04-23T21:58:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test -n 1 2024-04-23T21:58:56.633 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ return 2024-04-23T21:58:56.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' -z 1 ']' 2024-04-23T21:58:56.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ test '' = /tmp/tmp.8Tyc9aSFBO 2024-04-23T21:58:56.634 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ rm -Rf /tmp/tmp.8Tyc9aSFBO 2024-04-23T21:58:56.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ '[' 0 -eq 0 ']' 2024-04-23T21:58:56.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ echo OK 2024-04-23T21:58:56.707 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stdout:OK 2024-04-23T21:58:56.708 INFO:tasks.workunit.cluster1.client.mirror.smithi150.stderr:+ exit 0 2024-04-23T21:58:56.708 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-23T21:58:56.709 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp 2024-04-23T21:58:56.815 INFO:tasks.workunit:Stopping ['rbd/rbd_mirror_stress.sh'] on cluster1.client.mirror... 2024-04-23T21:58:56.815 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.cluster1.client.mirror /home/ubuntu/cephtest/clone.cluster1.client.mirror 2024-04-23T21:58:57.088 DEBUG:teuthology.parallel:result is None 2024-04-23T21:58:57.089 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror 2024-04-23T21:58:57.115 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror 2024-04-23T21:58:57.116 DEBUG:teuthology.orchestra.run.smithi150:> rmdir -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2024-04-23T21:58:57.176 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror 2024-04-23T21:58:57.177 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror-thrash 2024-04-23T21:58:57.188 INFO:tasks.rbd_mirror_thrash:joining rbd_mirror_thrash 2024-04-23T21:58:57.189 INFO:tasks.rbd_mirror_thrash:stat['kill'] = 20 2024-04-23T21:58:57.189 INFO:tasks.rbd_mirror_thrash:done joining 2024-04-23T21:58:57.189 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror-thrash 2024-04-23T21:58:57.210 INFO:tasks.rbd_mirror_thrash:joining rbd_mirror_thrash 2024-04-23T21:58:57.210 INFO:tasks.rbd_mirror_thrash:stat['kill'] = 17 2024-04-23T21:58:57.211 INFO:tasks.rbd_mirror_thrash:done joining 2024-04-23T21:58:57.211 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2024-04-23T21:58:57.220 DEBUG:tasks.rbd_mirror.cluster2.client.mirror.3:waiting for process to exit 2024-04-23T21:58:57.220 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T21:58:57.221 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.220+0000 7ff194258640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 74168) UID: 1000 2024-04-23T21:58:57.221 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.220+0000 7ff193a57640 -1 rbd::mirror::LeaderWatcher: 0x56184f37d200 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (2) No such file or directory 2024-04-23T21:58:57.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.221+0000 7ff193256640 -1 rbd::mirror::InstanceWatcher: 0x56184f174380 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.7454 object: (2) No such file or directory 2024-04-23T21:58:57.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.221+0000 7ff193a57640 -1 rbd::mirror::InstanceWatcher: 0x56184f678000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.7454 object: (2) No such file or directory 2024-04-23T21:58:57.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.221+0000 7ff193256640 -1 rbd::mirror::InstanceWatcher: 0x56184f175c00 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.7457 object: (2) No such file or directory 2024-04-23T21:58:57.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.221+0000 7ff18ca49640 -1 rbd::mirror::InstanceWatcher: 0x56184f174380 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:58:57.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.221+0000 7ff18d24a640 -1 rbd::mirror::InstanceWatcher: 0x56184f678000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:58:57.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.221+0000 7ff17f22e640 -1 rbd::mirror::InstanceWatcher: 0x56184f175c00 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:58:57.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.221+0000 7ff17fa2f640 -1 rbd::mirror::MirrorStatusUpdater 0x56184f3af560 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:57.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.221+0000 7ff193256640 -1 rbd::mirror::NamespaceReplayer: 0x56184f1c64e0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:58:57.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.221+0000 7ff186a3d640 -1 rbd::mirror::MirrorStatusUpdater 0x56184f1c8360 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:57.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.221+0000 7ff186a3d640 -1 rbd::mirror::MirrorStatusUpdater 0x56184f1c85a0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:57.225 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.222+0000 7ff193256640 -1 rbd::mirror::NamespaceReplayer: 0x56184f1c6000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:58:57.225 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.222+0000 7ff193256640 -1 rbd::mirror::NamespaceReplayer: 0x56184f1c61a0 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:58:57.225 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.222+0000 7ff18ca49640 -1 rbd::mirror::MirrorStatusUpdater 0x56184f1c8000 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:57.225 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.222+0000 7ff18ca49640 -1 rbd::mirror::MirrorStatusUpdater 0x56184f1c8120 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:57.225 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.222+0000 7ff193a57640 -1 rbd::mirror::NamespaceReplayer: 0x56184f1c6000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:58:57.225 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.222+0000 7ff193a57640 -1 rbd::mirror::NamespaceReplayer: 0x56184f1c61a0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:58:57.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.223+0000 7ff193a57640 -1 rbd::mirror::LeaderWatcher: 0x56184f0f0c00 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (2) No such file or directory 2024-04-23T21:58:57.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.224+0000 7ff193256640 -1 rbd::mirror::InstanceWatcher: 0x56184f174000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.7454 object: (2) No such file or directory 2024-04-23T21:58:57.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.224+0000 7ff18d24a640 -1 rbd::mirror::InstanceWatcher: 0x56184f174000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:58:57.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.224+0000 7ff18623c640 -1 rbd::mirror::MirrorStatusUpdater 0x56184ef505a0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:57.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.224+0000 7ff193256640 -1 rbd::mirror::NamespaceReplayer: 0x56184eddc1a0 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:58:57.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.224+0000 7ff18ca49640 -1 rbd::mirror::MirrorStatusUpdater 0x56184ef05560 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:57.227 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi150.stderr:2024-04-23T21:58:57.224+0000 7ff193a57640 -1 rbd::mirror::NamespaceReplayer: 0x56184eddc1a0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:58:57.237 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Stopped 2024-04-23T21:58:57.237 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2024-04-23T21:58:57.248 DEBUG:tasks.rbd_mirror.cluster2.client.mirror.2:waiting for process to exit 2024-04-23T21:58:57.248 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T21:58:57.279 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi150.stderr:2024-04-23T21:58:57.278+0000 7f95f1c38640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 107073) UID: 1000 2024-04-23T21:58:57.289 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Stopped 2024-04-23T21:58:57.289 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2024-04-23T21:58:57.300 DEBUG:tasks.rbd_mirror.cluster2.client.mirror.1:waiting for process to exit 2024-04-23T21:58:57.300 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T21:58:57.330 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.329+0000 7f90fb63f640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 104281) UID: 1000 2024-04-23T21:58:57.331 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.330+0000 7f90fa63d640 -1 rbd::mirror::InstanceWatcher: 0x5646c5b16380 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.9923 object: (2) No such file or directory 2024-04-23T21:58:57.331 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.330+0000 7f90fae3e640 -1 rbd::mirror::InstanceWatcher: 0x5646c6008000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.9923 object: (2) No such file or directory 2024-04-23T21:58:57.331 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.330+0000 7f90e6e16640 -1 rbd::mirror::InstanceWatcher: 0x5646c5b17c00 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:58:57.331 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.330+0000 7f90f4631640 -1 rbd::mirror::InstanceWatcher: 0x5646c5b16380 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:58:57.331 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.330+0000 7f90f4631640 -1 rbd::mirror::InstanceWatcher: 0x5646c6008000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:58:57.333 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.331+0000 7f90ede24640 -1 rbd::mirror::MirrorStatusUpdater 0x5646c5b6a480 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:57.333 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.331+0000 7f90ede24640 -1 rbd::mirror::MirrorStatusUpdater 0x5646c5b6a360 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:57.333 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.331+0000 7f90fa63d640 -1 rbd::mirror::NamespaceReplayer: 0x5646c5b68000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:58:57.333 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.331+0000 7f90fa63d640 -1 rbd::mirror::NamespaceReplayer: 0x5646c5b681a0 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:58:57.333 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.331+0000 7f90f3e30640 -1 rbd::mirror::MirrorStatusUpdater 0x5646c5b6a000 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:57.333 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.331+0000 7f90f3e30640 -1 rbd::mirror::MirrorStatusUpdater 0x5646c5b6a120 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:57.333 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.331+0000 7f90fae3e640 -1 rbd::mirror::NamespaceReplayer: 0x5646c5b68000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:58:57.334 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.331+0000 7f90fa63d640 -1 rbd::mirror::NamespaceReplayer: 0x5646c5b681a0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:58:57.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.332+0000 7f90fa63d640 -1 rbd::mirror::LeaderWatcher: 0x5646c5a80c00 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (2) No such file or directory 2024-04-23T21:58:57.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.332+0000 7f90fa63d640 -1 rbd::mirror::InstanceWatcher: 0x5646c5b16000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.9923 object: (2) No such file or directory 2024-04-23T21:58:57.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.332+0000 7f90f4631640 -1 rbd::mirror::InstanceWatcher: 0x5646c5b16000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:58:57.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.333+0000 7f90ed623640 -1 rbd::mirror::MirrorStatusUpdater 0x5646c58f05a0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:57.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.333+0000 7f90fa63d640 -1 rbd::mirror::NamespaceReplayer: 0x5646c6392000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:58:57.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.333+0000 7f90f3e30640 -1 rbd::mirror::MirrorStatusUpdater 0x5646c588b560 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:57.335 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi150.stderr:2024-04-23T21:58:57.333+0000 7f90fae3e640 -1 rbd::mirror::NamespaceReplayer: 0x5646c6392000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:58:57.345 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Stopped 2024-04-23T21:58:57.346 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2024-04-23T21:58:57.356 DEBUG:tasks.rbd_mirror.cluster2.client.mirror.0:waiting for process to exit 2024-04-23T21:58:57.356 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T21:58:57.387 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:57.386+0000 7f3e5ce38640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 78826) UID: 1000 2024-04-23T21:58:57.388 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:57.387+0000 7f3e5be36640 -1 rbd::mirror::InstanceWatcher: 0x560e1a1b2700 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.7858 object: (2) No such file or directory 2024-04-23T21:58:57.388 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:57.387+0000 7f3e5be36640 -1 rbd::mirror::InstanceWatcher: 0x560e1a1b2380 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.7858 object: (2) No such file or directory 2024-04-23T21:58:57.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:57.387+0000 7f3e55629640 -1 rbd::mirror::InstanceWatcher: 0x560e1a1b2700 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:58:57.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:57.387+0000 7f3e55629640 -1 rbd::mirror::InstanceWatcher: 0x560e1a1b2380 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:58:57.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:57.387+0000 7f3e55e2a640 -1 rbd::mirror::MirrorStatusUpdater 0x560e1a1f2120 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:57.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:57.387+0000 7f3e5be36640 -1 rbd::mirror::NamespaceReplayer: 0x560e1a1f01a0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:58:57.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:57.388+0000 7f3e4ee1c640 -1 rbd::mirror::MirrorStatusUpdater 0x560e1a1f25a0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:57.390 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:57.388+0000 7f3e5be36640 -1 rbd::mirror::NamespaceReplayer: 0x560e1a1f0000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:58:57.390 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:57.388+0000 7f3e55629640 -1 rbd::mirror::MirrorStatusUpdater 0x560e1a1f2000 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:57.390 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:57.388+0000 7f3e5c637640 -1 rbd::mirror::NamespaceReplayer: 0x560e1a1f0000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:58:57.390 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:57.388+0000 7f3e41601640 -1 rbd::mirror::PoolWatcher: 0x560e1a6a0960 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-04-23T21:58:58.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:58.529+0000 7f98854ae640 -1 rbd::mirror::LeaderWatcher: 0x555f64d8b200 handle_notify_heartbeat: error notifying heartbeat: (2) No such file or directory, releasing leader 2024-04-23T21:58:58.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:58.529+0000 7f9871486640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x555f65101680 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2024-04-23T21:58:58.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:58.529+0000 7f9871c87640 -1 rbd::mirror::PoolWatcher: 0x555f65b70000 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-04-23T21:58:58.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:58.529+0000 7f986b47a640 -1 rbd::mirror::PoolWatcher: 0x555f657aa780 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2024-04-23T21:58:58.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:58.530+0000 7f9885caf640 -1 rbd::mirror::LeaderWatcher: 0x555f64d8b200 handle_notify_lock_released: error notifying leader lock released: (2) No such file or directory 2024-04-23T21:58:58.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:58:58.530+0000 7f9885caf640 -1 rbd::mirror::LeaderWatcher: 0x555f64d8b200 handle_release_leader_lock: error releasing lock: (2) No such file or directory 2024-04-23T21:58:58.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.599+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.599+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.599+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.599+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.599+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.599+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.599+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.599+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.600+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.600+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.600+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.600+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.601+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.601+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.601+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.601+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.601+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.601+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.601+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.601+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.602+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.602+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.602+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.602+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.602+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.602+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.602+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.602+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.603+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.603+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.603+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.603+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.603+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.603+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.604+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.604+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.604+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.604+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.604+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.604+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.605+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.605+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.605+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.605+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.605+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.605+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.605+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.605+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.606+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.606+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.606+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.606+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.606+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.606+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.607+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.607+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.607+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.607+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.607+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.607+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.607+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.607+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.608+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.608+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.608+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.608+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.608+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.608+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.608+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.608+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.609+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.609+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.609+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.609+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.610+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.610+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.610+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.610+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.610+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.610+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.610+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.610+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.611+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.611+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.611+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.611+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.611+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.611+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.611+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.611+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.612+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.612+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.612+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.612+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.612+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.612+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.612+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.612+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.613+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.613+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.613+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.613+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.614+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.614+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.614+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.614+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.614+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.614+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.614+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.614+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.615+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.615+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.615+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.615+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.615+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.615+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.615+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.616+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.616+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.616+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.616+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.616+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.617+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.617+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.617+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.617+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.617+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.617+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.617+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.617+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.618+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.618+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.618+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.618+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.618+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.618+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.618+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.618+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.618+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.618+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.619+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.619+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.619+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.619+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.619+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.619+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.619+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.619+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.620+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.620+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.620+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.620+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.620+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.620+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.620+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.620+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.621+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.621+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.621+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.621+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.621+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.621+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.622+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.622+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.622+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.622+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.622+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.622+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.623+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.623+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.623+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.623+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.623+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.623+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.623+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.623+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.623+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.623+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.624+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.624+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.624+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.624+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.624+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.624+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.624+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.624+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.625+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.625+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.625+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.625+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.625+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.625+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.626+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.626+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.626+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.626+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.626+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.626+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.626+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.626+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.627+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.627+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.627+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.627+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.627+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.627+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.627+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.627+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.627+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.627+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.628+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.628+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.628+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.628+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.628+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.628+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.628+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.628+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.628+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.628+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.628+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.628+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.629+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.629+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.629+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.629+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.629+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.629+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.629+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.629+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.630+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.630+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.630+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.630+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.630+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.630+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.631+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.631+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.631+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.631+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.631+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.631+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.632+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.632+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.632+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.632+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.633+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.633+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.634+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.634+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.634+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.634+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.634+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.634+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.635+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.635+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.635+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.635+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.635+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.635+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.635+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.635+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.636+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.636+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.636+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.636+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.636+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.636+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.637+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.637+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.637+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.637+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.637+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.637+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.638+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.638+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.638+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.638+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.638+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.638+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.638+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.638+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.639+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.639+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.639+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.639+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.639+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.639+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.639+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.639+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.640+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.640+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.640+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.640+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.640+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.640+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.641+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.641+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.641+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.641+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.641+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.641+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.641+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.641+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.642+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.642+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.642+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.642+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.642+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.642+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.643+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.643+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.643+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.643+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.643+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.643+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.643+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.643+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.644+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.644+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.644+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.644+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.644+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.644+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.645+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.645+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.645+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.645+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.645+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.645+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.646+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.646+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.646+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.646+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.646+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.646+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.647+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.647+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.647+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.647+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.647+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.647+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.647+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.647+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.648+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.648+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.648+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.648+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.648+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.648+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.648+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.648+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.648+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.648+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.649+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.649+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.649+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.649+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.649+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.649+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.649+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.649+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.650+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.650+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.650+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.650+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.650+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.650+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.650+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.650+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.651+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.651+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.651+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.651+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.651+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.651+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.652+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.652+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.652+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.652+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.652+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.652+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.652+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.652+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.653+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.653+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.653+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.653+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.654+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.654+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.654+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.654+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.654+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.654+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.654+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.654+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.655+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.655+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.655+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.655+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.655+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.655+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.655+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.655+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.656+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.656+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.656+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.656+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.656+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.656+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.657+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.657+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.657+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.657+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.657+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.657+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.657+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.657+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.658+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.658+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.658+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.658+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.658+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.658+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.658+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.658+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.659+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.659+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.659+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.659+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.659+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.659+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.659+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.659+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.660+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.660+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.660+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.660+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.660+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.660+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.661+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.661+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.661+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.661+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.662+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.662+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.662+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.662+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.662+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.662+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.663+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.663+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.663+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.663+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.663+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.663+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.664+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.664+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.664+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.664+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.664+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.664+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.664+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.664+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.665+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.665+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.665+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.665+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.665+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.665+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.665+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.665+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.666+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.666+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.666+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.666+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.666+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.666+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.666+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.666+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.667+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.667+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.667+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.667+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.667+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.667+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.667+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.667+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.668+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.668+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.668+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.668+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.668+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.668+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.668+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.668+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.669+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.669+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.669+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.669+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.669+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.669+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.669+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.669+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.670+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.670+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.670+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.670+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.670+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.670+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.670+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.670+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.671+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.671+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.671+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.671+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.671+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.671+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.672+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.672+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.672+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.672+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.672+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.672+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.672+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.672+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.672+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.672+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.673+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.673+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.673+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.673+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.673+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.673+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.674+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.674+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.674+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.674+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.674+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.674+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.674+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.674+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.675+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.675+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.676+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.676+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.676+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.676+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.676+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.676+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.676+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.676+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.677+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.677+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.677+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.677+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.677+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.677+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.677+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.677+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.678+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.678+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.678+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.678+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.678+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.678+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.678+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.679+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.679+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.679+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.680+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.680+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.681+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.681+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.681+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.681+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.682+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.682+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.682+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.682+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.682+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.682+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.682+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.682+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.683+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.683+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.683+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.683+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.683+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.683+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.683+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.683+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.684+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.684+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.684+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.684+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.684+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.684+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.684+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.684+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.685+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.685+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.685+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.685+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.686+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.686+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.686+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.686+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.686+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.686+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.687+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.687+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.687+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.687+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.687+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.687+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.687+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.687+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.688+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.688+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.688+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.688+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.689+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.689+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.689+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.689+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.689+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.689+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.689+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.689+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.690+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.690+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.690+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.690+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.691+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.691+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.691+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.691+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.692+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.692+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.692+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.692+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.692+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.692+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.692+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.692+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.693+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.693+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.693+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.693+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.693+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.693+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.693+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.693+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.693+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.693+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.694+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.694+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.694+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.694+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.694+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.694+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.694+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.694+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.695+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.695+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.695+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.695+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.695+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.695+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.696+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.696+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.696+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.696+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.697+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.697+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.697+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.697+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.697+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.697+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.697+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.697+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.698+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.698+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.698+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.698+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.698+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.698+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.698+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.698+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.699+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.699+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.699+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.699+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.699+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.699+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.699+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.699+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.700+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.700+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.700+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.700+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.700+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.700+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.700+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.700+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.701+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.701+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.701+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.701+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.701+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.701+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.701+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.701+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.702+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.702+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.702+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.702+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.702+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.702+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.702+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.702+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.703+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.703+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.703+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.703+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.703+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.703+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.703+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.703+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.704+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.704+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.704+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.704+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.704+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.704+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.704+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.704+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.705+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.705+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.705+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.705+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.706+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.706+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.706+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.706+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.706+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.706+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.706+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.706+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.707+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.707+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.707+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.707+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.707+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.707+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.707+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.707+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.708+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.708+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.708+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.708+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.708+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.708+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.708+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.708+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.709+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.709+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.709+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.709+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.709+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.709+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.709+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.709+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.710+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.710+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.710+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.710+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.710+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.710+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.710+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.710+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.711+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.711+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.711+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.711+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.711+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.711+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.711+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.711+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.711+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.711+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.711+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.711+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.712+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.712+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.712+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.712+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.712+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.712+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.712+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.712+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.713+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.713+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.713+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.713+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.713+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.713+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.713+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.714+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.714+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.714+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.714+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.715+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.715+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.715+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.715+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.715+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.715+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.715+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.715+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.716+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.716+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.716+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.716+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.716+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.716+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.716+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.716+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.717+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.717+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.717+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.717+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.717+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.717+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.717+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.717+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.718+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.718+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.718+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.718+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.719+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.719+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.719+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.719+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.719+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.719+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.719+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.719+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.719+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.719+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.720+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.720+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.720+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.720+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.720+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.720+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.720+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.720+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.721+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.721+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.721+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.721+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.721+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.721+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.721+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.721+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.722+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.722+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.722+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.722+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.722+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.722+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.722+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.722+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.723+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.723+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.723+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.723+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.723+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.723+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.723+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.723+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.724+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.724+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.724+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.724+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.724+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.724+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.724+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.724+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.725+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.725+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.725+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.725+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.725+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.725+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.725+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.725+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.726+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.726+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.726+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.726+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.726+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.726+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.726+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.726+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.727+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.727+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.727+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.727+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.727+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.727+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.727+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.727+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.728+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.728+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.728+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.728+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.728+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.728+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.728+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.728+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.729+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.729+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.729+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.729+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.730+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.730+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.730+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.730+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.730+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.730+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.730+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.730+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.731+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.731+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.731+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.731+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.731+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.731+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.731+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.731+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.732+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.732+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.732+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.732+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.732+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.732+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.732+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.732+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.732+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.732+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.733+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.733+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.733+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.733+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.733+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.733+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.733+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.733+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.733+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.733+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.734+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.734+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.734+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.734+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.734+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.734+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.735+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.735+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.735+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.735+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.735+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.735+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.736+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.736+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.736+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.736+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.737+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.737+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.737+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.737+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.737+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.737+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.737+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.737+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.738+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.738+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.738+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.738+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.738+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.738+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.738+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.738+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.739+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.739+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.739+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.739+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.739+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.739+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.739+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.739+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.740+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.740+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.740+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.740+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.740+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.740+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.741+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.741+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.741+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.741+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.741+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.741+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.742+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.742+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.742+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.742+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.742+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.742+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.742+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.742+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.743+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.743+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.743+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.743+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.743+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.743+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.743+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.743+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.744+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.744+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.744+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.744+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.744+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.744+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.745+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.745+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.745+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.745+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.745+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.745+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.745+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.745+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.746+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.746+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.746+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.746+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.746+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.746+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.746+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.746+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.747+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.747+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.747+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.747+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.747+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.747+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.747+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.747+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.748+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.748+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.748+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.748+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.749+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.749+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.749+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.749+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.749+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.749+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.749+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.749+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.750+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.750+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.750+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.750+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.750+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.750+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.750+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.750+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.751+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.751+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.751+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.751+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.751+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.751+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.751+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.751+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.752+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.752+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.752+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.752+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.753+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.753+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.753+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.753+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.753+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.753+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.753+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.753+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.754+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.754+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.754+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.754+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.754+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.754+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.754+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.754+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.755+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.755+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.755+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.755+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.755+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.755+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.755+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.755+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.756+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.756+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.756+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.756+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.756+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.756+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.756+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.756+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.757+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.757+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.757+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.757+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.758+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.758+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.758+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.758+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.758+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.758+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.758+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.758+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.759+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.759+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.759+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.759+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.759+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.759+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.759+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.759+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.760+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.760+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.760+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.760+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.760+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.760+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.761+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.761+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.761+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.761+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.762+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.762+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.762+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.762+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.762+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.762+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.762+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.762+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.763+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.763+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.763+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.763+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.763+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.763+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.763+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.763+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.763+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.763+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.764+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.764+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.764+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.764+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.764+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.764+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.764+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.764+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.765+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.765+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.765+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.765+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.765+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.765+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.765+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.765+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.766+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.766+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.766+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.766+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.766+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.766+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.766+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.766+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.767+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.767+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.767+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.767+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.767+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.767+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.767+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.767+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.767+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.767+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.768+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.768+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.768+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.768+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.768+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.768+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.768+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.768+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.769+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.769+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.769+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.769+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.769+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.769+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.769+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.769+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.770+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.770+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.770+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.770+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.770+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.770+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.770+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.770+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.770+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.770+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.771+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.771+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.771+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.771+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.772+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.772+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.772+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.772+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.772+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.772+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.772+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.772+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.772+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.772+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.773+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.773+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.773+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.773+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.773+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.773+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.773+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.773+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.774+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.774+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.774+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.774+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.774+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.774+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.774+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.774+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.774+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.774+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.775+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.775+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.775+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.775+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.775+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.775+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.775+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.775+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.776+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.776+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.776+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.776+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.776+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.776+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.776+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.776+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.776+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.776+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.777+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.777+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.777+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.777+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.777+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.777+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.777+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.777+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.777+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.777+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.778+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.778+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.778+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.778+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.778+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.778+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.778+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.778+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.779+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.779+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.779+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.779+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.780+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.780+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.780+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.780+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.780+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.780+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.780+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.780+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.781+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.781+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.781+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.781+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.782+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.782+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.782+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.782+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.782+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.782+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.782+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.782+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.783+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.783+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.783+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.783+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.783+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.783+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.783+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.783+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.784+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.784+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.784+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.784+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.784+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.784+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.784+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.784+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.785+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.785+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.785+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.785+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.785+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.785+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.785+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.785+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.786+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.786+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.786+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.786+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.786+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.786+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.786+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.786+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.787+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.787+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.787+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.787+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.787+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.787+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.787+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.787+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.788+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.788+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.788+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.788+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.788+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.788+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.788+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.788+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.789+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.789+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.789+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.789+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.789+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.789+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.789+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.789+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.790+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.790+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.790+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.790+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.790+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.790+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.790+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.790+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.790+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.790+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.791+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.791+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.791+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.791+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.791+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.791+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.791+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.791+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.792+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.792+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.792+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.792+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.793+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.793+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.793+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.793+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.793+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.793+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.793+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.793+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.794+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.794+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.794+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.794+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.794+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.794+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.794+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.794+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.794+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.794+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.795+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.795+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.795+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.795+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.795+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.795+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.795+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.795+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.796+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.796+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.796+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.796+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.796+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.796+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.796+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.796+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.796+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.796+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.797+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.797+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.797+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.797+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.797+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.797+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.797+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.797+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.798+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.798+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.798+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.798+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.798+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.798+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.798+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.798+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.799+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.799+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.799+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.799+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.800+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.800+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.800+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.800+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.801+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.801+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.801+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.801+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.801+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.801+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.801+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.801+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.802+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.802+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.802+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.802+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.802+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.802+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.802+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.802+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.803+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.803+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.803+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.803+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.803+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.803+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.803+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.803+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.803+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.803+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.804+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.804+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.804+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.804+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.804+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.804+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.805+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.805+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.805+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.805+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.805+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.805+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.805+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.805+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.806+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.806+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.806+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.806+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.806+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.806+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.806+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.806+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.807+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.808+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.808+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.808+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.808+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.808+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.808+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.808+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.808+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.808+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.809+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.809+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.809+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.809+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.809+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.809+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.810+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.810+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.811+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.811+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.811+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.811+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.811+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.811+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.811+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.811+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.812+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.812+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.812+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.812+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.812+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.812+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.812+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.812+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.813+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.813+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.813+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.813+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.813+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.813+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.813+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.813+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.814+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.814+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.814+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.814+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.814+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.814+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.815+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.815+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.815+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.815+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.815+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.815+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:58.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.815+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:58.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.815+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.816+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.816+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.816+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.816+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.816+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.816+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.816+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.816+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.817+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.817+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.817+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.817+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.817+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.817+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.817+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.817+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.818+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.818+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.818+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.818+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.819+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.819+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.819+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.819+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.819+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.819+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.819+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.819+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.820+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.820+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.820+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.820+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.820+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.820+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.820+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.820+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.821+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.821+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.821+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.821+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.821+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.821+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.821+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.821+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.822+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.822+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.822+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.822+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.822+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.822+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.822+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.822+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.823+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.823+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.823+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.823+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.823+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.823+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.823+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.823+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.823+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.823+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.824+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.824+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.824+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.824+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.824+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.824+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.824+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.824+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.824+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.825+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.825+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.825+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.825+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.825+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.825+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.825+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.826+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.826+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.826+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.826+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.826+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.826+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.826+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.826+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.827+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.827+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.827+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.827+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.827+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.827+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.827+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.827+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.828+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.828+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.828+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.828+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.828+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.828+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.828+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.828+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.829+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.829+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.829+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.829+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.829+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.829+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.830+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.830+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.830+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.830+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.830+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.830+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.830+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.830+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.831+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.831+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.831+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.831+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.832+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.832+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.832+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.832+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.832+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.832+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.833+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.833+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.833+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.833+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.834+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.834+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.834+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.834+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.834+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.834+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.834+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.834+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.834+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.834+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.835+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.835+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.835+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.835+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.835+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.835+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.836+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.836+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.836+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.836+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.836+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.836+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.836+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.836+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.836+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.836+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.837+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.837+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.838+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.838+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.838+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.838+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.838+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.838+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.838+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.838+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.839+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.839+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.839+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.839+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.839+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.839+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.839+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.839+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.840+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.840+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.840+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.840+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.840+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.840+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.840+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.840+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.840+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.840+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.841+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.841+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.842+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.842+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.842+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.842+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.842+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.842+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.842+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.842+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.843+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.843+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.843+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.843+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.843+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.843+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.843+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.843+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.844+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.844+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.844+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.844+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.844+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.844+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.844+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.844+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.845+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.845+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.845+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.845+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.846+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.846+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.846+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.846+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.847+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.847+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.847+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.847+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.847+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.847+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.847+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.847+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.848+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.848+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.848+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.848+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.848+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.848+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.848+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.848+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.849+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.849+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.849+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.849+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.850+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.850+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.850+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.850+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.850+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.850+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.850+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.850+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.851+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.851+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.851+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.851+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.851+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.851+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.851+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.851+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.852+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.852+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.852+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.852+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.852+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.852+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.853+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.853+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.853+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.853+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.854+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.854+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.854+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.854+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.854+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.854+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.854+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.854+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.855+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.855+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.855+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.855+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.856+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.856+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.856+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.856+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.856+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.856+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.857+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.857+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.857+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.857+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.857+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.857+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.857+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.857+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.857+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.857+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.858+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.858+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.858+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.858+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.859+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.859+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.859+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.859+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.859+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.859+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.859+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.859+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.860+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.860+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.860+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.860+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.860+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.860+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.860+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.860+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.861+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.861+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.861+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.861+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.861+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.861+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.861+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.861+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.862+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.862+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.862+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.862+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.862+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.862+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.862+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.862+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.863+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.863+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.863+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.863+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.863+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.863+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.863+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.863+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.864+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.864+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.864+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.864+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.864+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.864+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.864+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.864+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.865+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.865+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.865+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.865+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.865+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.865+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.865+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.865+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.866+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.866+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.866+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.866+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.867+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.867+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.867+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.867+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.867+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.867+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.867+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.867+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.868+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.868+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.868+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.868+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.868+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.868+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.869+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.869+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.869+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.869+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.869+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.869+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.870+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.870+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.870+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.870+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.870+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.870+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.870+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.870+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.870+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.870+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.871+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.871+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.871+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.871+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.871+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.871+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.871+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.871+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.872+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.872+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.872+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.872+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.872+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.872+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.872+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.872+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.873+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.873+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.873+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.873+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.873+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.873+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.873+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.873+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.874+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.874+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.874+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.874+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.874+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.874+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.874+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.874+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.875+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.875+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.875+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.875+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.875+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.875+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.875+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.875+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.876+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.876+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.876+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.876+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.876+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.876+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.876+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.876+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.877+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.877+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.877+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.877+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.877+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.877+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.877+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.877+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.878+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.878+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.878+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.878+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.878+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.878+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.878+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.878+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.879+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.879+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.879+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.879+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.879+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.879+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.879+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.879+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.880+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.880+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.880+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.880+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.880+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.880+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.880+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.880+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.881+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.881+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.881+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.881+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.881+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.881+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.881+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.881+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.882+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.882+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.882+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.882+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.883+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.883+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.883+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.883+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.883+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.883+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.883+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.883+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.884+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.884+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.884+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.885+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.885+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.885+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.885+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.885+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.885+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.885+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.885+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.885+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.885+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.885+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.886+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.886+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.886+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.886+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.886+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.886+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.886+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.886+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.886+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.887+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.887+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.887+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.887+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.887+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.887+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.887+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.887+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.888+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.888+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.888+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.888+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.888+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.888+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.889+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.889+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.889+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.889+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.889+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.889+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.889+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.889+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.889+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.889+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.890+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.890+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.890+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.890+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.891+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.891+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.891+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.891+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.891+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.891+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.892+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.892+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.892+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.892+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.892+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.892+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.892+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.892+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.893+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.893+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.893+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.893+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.893+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.893+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.893+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.893+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.894+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.894+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.894+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.894+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.894+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.894+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.894+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.894+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.894+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.894+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.895+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.895+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.895+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.895+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.895+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.895+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.895+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.895+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.896+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.896+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.896+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.896+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.896+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.896+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.896+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.896+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.897+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.897+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.897+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.897+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.897+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.897+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.897+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.897+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.898+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.898+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.898+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.898+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.898+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.898+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.898+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.898+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.899+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.899+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.899+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.899+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.899+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.899+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.899+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.899+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.900+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.900+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.900+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.900+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.900+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.900+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.900+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.900+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.901+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.901+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.901+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.901+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.901+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.901+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.901+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.901+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.902+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.902+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.902+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.902+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.902+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.902+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.902+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.902+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.903+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.903+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.903+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.903+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.903+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.903+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.904+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.904+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.904+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.904+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.904+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.904+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.904+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.904+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.905+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.905+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.905+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.905+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.905+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.905+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.905+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.905+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.906+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.906+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.906+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.906+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.906+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.906+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.906+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.906+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.907+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.907+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.907+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.907+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.907+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.907+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.907+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.907+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.908+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.908+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.908+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.908+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.908+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.908+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.908+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.908+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.909+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.909+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.909+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.909+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.909+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.909+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.909+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.909+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.910+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.910+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.910+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.910+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.910+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.910+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.910+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.910+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.911+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.911+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.911+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.911+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.911+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.911+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.911+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.911+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.912+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.912+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.912+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.912+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.913+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.913+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.913+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.913+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.913+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.913+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.913+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.913+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.914+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.914+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.914+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.914+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.914+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.914+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.914+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.914+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.915+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.915+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.915+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.915+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.915+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.915+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.916+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.916+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.916+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.916+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.916+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.916+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.916+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.916+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.917+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.917+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.917+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.917+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.917+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.917+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.917+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.917+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.918+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.918+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.918+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.918+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.918+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.918+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.918+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.918+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.919+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.919+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.919+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.919+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.919+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.919+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.919+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.919+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.920+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.920+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.920+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.920+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.920+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.920+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.920+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.920+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.921+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.921+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.921+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.921+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.921+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.921+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.921+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.922+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.922+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.922+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.922+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.922+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.922+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.922+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.923+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.923+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.923+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.923+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.923+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.923+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.923+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.923+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.924+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.924+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.924+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.924+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.925+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.925+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.925+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.925+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.926+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.926+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.926+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.926+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.926+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.926+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.926+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.926+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.926+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.926+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.927+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.927+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.927+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.927+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.927+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.927+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.927+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.927+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.928+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.928+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.928+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.928+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.928+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.928+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.928+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.928+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.929+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.929+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.929+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.929+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.929+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.929+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.929+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.929+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.930+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.930+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.930+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.930+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.930+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.930+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.930+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.931+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.931+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.931+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.932+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.932+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.932+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.932+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.932+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.932+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.932+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.932+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.933+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.933+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.933+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.933+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.933+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.933+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.933+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.933+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.934+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.934+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.934+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.934+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.934+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.934+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.934+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.934+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.935+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.935+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.935+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.935+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.935+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.935+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.935+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.935+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.936+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.936+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.936+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.936+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.936+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.936+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.936+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.936+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.937+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.937+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.937+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.937+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.937+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.937+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.937+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.937+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.938+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.938+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.938+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.938+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.938+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.938+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.938+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.938+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.938+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.938+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.938+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.938+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.939+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.939+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.939+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.939+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.939+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.939+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.939+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.939+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.940+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.940+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.940+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.940+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.940+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.940+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.940+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.940+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.940+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.940+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.940+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.940+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.941+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.941+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.941+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.941+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.941+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.941+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.941+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.941+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.942+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.942+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.942+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.942+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.942+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.942+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.943+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.943+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.943+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.943+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.943+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.943+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.943+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.943+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.944+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.944+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.944+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.944+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.944+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.944+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.944+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.944+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.945+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.945+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.945+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.945+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.945+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.945+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.945+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.945+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.946+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.946+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.946+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.946+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.946+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.946+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.946+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.946+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.947+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.947+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.947+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.947+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.947+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.947+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.947+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.947+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.948+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.948+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.948+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.948+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.948+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.948+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.948+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.948+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.949+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.949+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.949+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.949+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.949+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.949+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.949+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.949+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.950+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.950+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.950+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.950+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.950+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.950+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.950+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.950+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.950+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.950+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.951+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.951+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.951+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.951+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.951+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.951+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.951+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.951+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.952+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.952+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.952+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.952+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.952+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.952+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.952+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.953+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.952+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.953+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.953+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.953+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.953+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.953+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.953+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.953+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.953+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.953+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.954+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.954+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.954+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.954+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.954+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.954+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.954+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.954+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.955+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.955+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.955+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.955+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.956+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.956+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.956+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.956+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.956+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.956+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.956+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.956+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.957+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.957+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.957+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.957+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.957+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.957+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.957+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.957+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.958+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.958+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.958+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.958+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.958+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.958+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.958+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.958+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.959+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.959+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.959+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.959+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.959+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.959+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.959+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.959+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.960+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.960+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.960+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.960+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.961+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.961+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.961+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.961+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.961+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.961+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.961+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.961+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.962+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.962+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.962+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.962+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.962+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.962+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.962+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.962+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.963+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.963+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.963+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.963+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.963+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.963+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.963+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.963+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.964+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.964+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.964+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.964+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.965+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.965+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.965+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.965+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.965+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.965+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.965+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.965+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.966+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.966+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.966+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.966+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.966+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.966+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.966+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.966+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.967+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.967+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.967+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.967+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.967+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.967+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.968+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.968+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.968+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.968+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.969+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.969+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.969+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.969+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.969+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.969+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.969+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.969+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.969+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.969+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.970+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.970+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.970+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.970+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.970+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.970+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.970+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.970+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.971+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.971+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.971+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.971+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.971+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.971+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.971+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.971+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.971+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.971+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.971+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.971+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.972+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.972+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.972+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.972+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.973+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.973+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.973+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.973+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.973+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.973+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.974+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.974+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.974+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.974+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.974+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.974+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.975+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.975+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.975+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.975+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.975+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.975+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.975+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.975+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.975+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.975+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.976+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.976+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.976+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.976+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.977+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.977+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.977+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.977+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.977+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.977+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.977+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.977+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.978+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.978+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.978+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.978+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.978+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.978+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.978+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.978+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.979+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.979+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.979+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.979+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.979+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.979+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.979+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.979+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.979+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.979+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.980+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.980+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.980+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.980+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.980+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.980+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.980+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.980+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.981+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.981+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.981+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.981+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.981+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.981+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.981+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.981+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.982+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.982+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.982+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.982+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.982+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.982+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.982+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.982+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.982+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.982+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.983+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.983+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.983+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.983+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.983+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.983+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.983+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.983+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.984+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.984+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.984+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.984+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.984+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.984+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.984+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.984+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.984+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.984+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.985+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.985+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.985+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.985+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.985+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.985+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.986+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.986+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.986+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.986+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.986+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.986+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.986+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.986+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.986+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.986+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.987+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.987+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.987+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.987+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.987+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.987+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.987+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.987+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.988+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.988+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.988+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.988+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.988+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.988+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.988+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.988+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.989+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.989+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.989+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.989+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.989+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.989+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.990+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.990+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.990+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.990+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.990+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.990+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.990+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.990+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.991+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.991+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.991+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.991+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.991+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.991+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.991+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.991+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.992+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.992+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.992+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.992+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.992+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.992+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.993+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.993+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.993+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.993+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.993+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.993+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.993+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.993+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.993+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.993+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.993+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.993+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.994+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.994+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.994+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.994+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.994+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.994+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.994+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.994+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.995+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.995+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.995+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.995+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.995+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.995+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.995+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.995+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.996+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.996+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.996+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.996+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.997+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.997+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.997+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.997+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.997+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.997+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.998+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.998+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.998+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.998+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.998+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.998+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.998+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.998+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.998+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.998+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.999+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.999+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.999+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.999+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:58.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.999+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:58.999+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.999+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:58.999+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.000+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.000+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.000+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.000+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.000+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.000+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.000+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.000+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.000+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.000+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.001+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.001+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.001+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.001+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.001+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.001+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.001+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.001+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.002+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.002+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.002+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.002+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.002+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.002+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.003+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.003+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.003+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.003+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.003+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.003+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.004+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.004+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.004+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.004+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.004+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.004+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.004+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.004+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.005+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.005+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.005+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.005+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.005+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.005+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.005+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.005+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.006+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.006+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.006+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.006+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.006+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.006+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.006+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.006+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.007+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.007+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.007+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.007+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.007+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.007+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.007+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.007+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.008+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.008+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.008+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.008+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.008+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.008+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.008+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.008+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.009+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.009+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.009+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.009+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.009+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.009+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.010+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.010+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.010+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.010+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.010+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.010+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.011+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.011+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.011+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.011+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.011+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.011+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.012+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.012+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.012+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.012+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.012+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.012+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.012+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.012+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.013+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.013+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.013+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.013+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.013+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.013+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.014+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.014+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.014+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.014+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.015+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.015+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.016+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.016+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.016+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.016+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.016+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.016+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.016+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.016+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.017+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.017+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.017+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.017+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.018+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.018+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.018+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.018+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.018+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.018+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.018+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.018+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.019+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.019+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.019+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.019+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.019+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.019+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.019+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.019+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.019+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.019+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.020+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.020+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.020+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.020+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.021+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.021+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.021+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.021+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.021+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.021+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.021+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.022+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.022+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.022+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.022+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.022+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.022+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.022+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.023+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.023+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.023+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.023+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.023+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.023+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.024+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.024+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.024+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.024+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.024+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.024+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.024+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.024+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.025+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.025+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.025+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.025+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.025+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.025+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.025+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.025+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.026+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.026+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.026+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.026+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.026+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.026+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.026+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.026+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.027+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.027+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.027+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.027+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.027+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.027+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.027+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.027+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.028+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.028+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.028+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.028+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.028+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.028+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.028+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.028+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.028+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.028+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.029+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.029+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.029+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.029+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.029+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.029+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.030+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.030+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.030+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.030+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.030+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.030+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.030+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.030+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.030+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.030+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.031+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.031+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.031+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.031+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.031+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.031+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.031+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.031+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.032+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.032+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.032+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.032+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.032+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.032+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.032+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.032+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.033+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.033+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.033+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.033+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.033+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.033+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.034+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.034+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.034+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.034+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.035+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.035+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.035+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.035+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.035+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.035+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.035+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.035+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.036+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.036+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.036+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.036+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.036+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.036+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.036+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.036+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.037+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.037+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.037+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.037+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.037+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.037+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.037+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.037+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.037+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.037+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.037+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.037+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.038+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.038+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.038+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.038+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.038+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.038+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.038+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.038+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.038+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.038+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.039+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.039+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.039+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.039+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.039+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.039+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.040+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.040+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.040+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.040+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.040+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.040+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.040+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.040+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.040+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.040+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.041+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.041+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.041+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.041+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.041+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.041+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.042+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.042+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.042+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.042+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.042+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.042+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.043+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.043+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.043+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.043+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.043+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.043+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.043+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.043+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.044+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.044+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.044+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.044+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.044+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.044+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.045+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.045+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.045+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.045+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.045+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.045+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.045+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.045+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.045+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.045+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.046+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.046+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.046+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.046+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.046+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.046+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.046+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.046+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.047+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.047+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.047+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.047+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.047+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.047+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.047+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.047+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.048+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.048+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.048+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.048+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.048+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.048+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.049+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.049+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.049+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.049+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.049+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.049+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.049+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.049+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.050+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.050+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.050+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.050+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.050+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.050+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.051+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.051+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.051+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.051+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.051+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.051+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.051+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.051+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.051+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.051+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.051+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.051+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.052+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.052+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.052+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.052+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.052+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.052+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.053+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.053+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.053+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.053+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.053+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.053+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.053+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.053+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.054+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.054+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.054+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.054+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.054+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.054+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.055+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.055+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.055+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.055+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.055+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.055+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.055+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.055+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.056+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.056+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.056+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.056+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.056+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.056+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.056+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.056+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.057+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.057+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.057+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.057+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.057+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.057+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.057+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.057+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.058+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.058+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.058+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.058+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.058+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.058+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.059+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.059+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.059+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.059+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.060+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.060+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.060+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.060+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.060+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.060+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.061+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.061+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.061+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.061+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.061+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.061+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.061+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.061+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.062+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.062+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.062+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.062+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.062+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.062+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.062+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.062+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.063+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.063+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.063+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.063+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.063+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.063+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.064+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.064+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.064+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.064+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.064+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.064+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.064+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.064+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.065+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.065+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.065+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.065+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.065+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.065+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.066+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.066+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.066+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.066+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.066+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.066+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.066+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.066+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.067+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.067+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.067+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.067+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.067+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.067+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.067+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.067+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.067+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.067+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.067+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.067+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.068+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.068+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.068+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.068+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.068+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.068+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.068+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.068+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.068+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.068+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.069+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.069+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.069+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.069+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.069+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.069+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.069+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.069+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.069+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.069+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.070+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.070+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.070+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.070+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.070+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.070+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.070+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.070+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.070+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.070+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.071+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.071+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.071+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.071+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.071+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.071+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.072+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.072+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.072+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.072+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.072+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.072+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.072+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.072+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.072+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.072+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.072+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.072+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.073+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.073+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.073+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.073+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.073+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.073+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.073+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.073+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.074+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.074+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.074+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.074+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.074+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.074+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.075+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.075+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.075+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.075+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.076+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.076+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.076+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.076+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.076+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.076+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.076+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.076+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.077+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.077+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.077+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.077+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.077+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.077+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.078+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.078+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.078+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.078+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.078+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.078+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.078+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.078+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.079+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.079+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.079+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.079+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.079+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.079+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.079+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.079+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.079+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.079+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.080+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.080+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.080+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.080+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.080+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.080+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.080+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.080+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.080+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.080+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.081+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.081+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.081+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.081+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.081+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.081+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.081+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.081+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.081+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.081+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.082+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.082+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.082+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.082+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.082+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.082+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.082+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.082+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.083+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.083+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.083+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.083+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.083+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.083+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.083+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.083+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.084+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.084+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.084+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.084+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.084+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.084+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.084+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.084+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.085+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.085+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.085+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.085+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.085+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.085+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.085+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.085+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.086+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.086+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.086+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.086+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.086+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.086+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.086+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.086+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.087+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.087+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.087+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.087+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.087+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.087+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.087+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.087+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.088+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.088+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.088+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.088+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.088+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.088+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.088+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.088+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.088+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.088+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.089+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.089+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.090+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.090+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.090+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.090+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.090+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.090+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.090+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.090+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.091+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.091+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.091+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.091+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.091+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.091+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.091+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.091+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.091+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.091+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.091+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.091+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.092+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.092+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.092+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.092+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.092+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.092+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.092+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.092+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.092+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.092+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.092+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.092+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.093+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.093+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.093+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.093+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.093+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.093+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.093+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.093+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.094+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.094+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.094+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.094+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.094+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.094+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.094+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.094+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.095+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.095+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.095+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.095+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.095+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.095+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.096+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.096+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.096+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.096+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.097+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.097+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.097+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.097+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.097+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.097+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.097+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.097+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.097+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.097+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.097+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.097+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.098+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.098+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.098+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.098+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.098+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.098+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.098+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.098+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.099+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.099+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.099+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.099+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.099+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.099+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.100+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.100+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.100+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.100+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.100+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.101+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.101+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.101+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.102+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.102+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.102+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.102+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.102+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.102+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.103+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.103+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.103+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.103+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.103+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.103+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.103+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.103+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.104+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.104+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.104+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.104+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.104+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.104+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.105+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.105+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.105+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.105+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.105+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.105+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.105+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.105+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.106+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.106+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.106+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.106+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.106+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.106+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.106+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.106+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.106+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.106+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.107+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.107+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.107+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.107+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.107+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.107+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.107+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.107+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.108+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.108+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.108+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.108+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.108+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.108+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.109+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.109+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.109+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.109+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.110+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.110+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.110+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.110+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.110+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.110+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.110+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.110+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.110+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.110+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.111+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.111+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.111+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.111+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.111+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.111+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.111+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.111+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.112+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.112+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.112+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.112+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.112+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.112+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.113+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.113+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.113+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.113+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.113+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.113+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.113+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.113+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.114+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.114+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.114+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.114+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.114+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.114+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.114+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.114+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.115+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.115+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.115+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.115+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.115+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.115+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.115+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.115+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.116+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.116+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.116+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.116+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.116+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.116+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.116+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.116+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.116+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.116+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.117+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.117+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.117+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.117+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.117+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.117+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.118+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.118+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.118+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.118+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.119+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.119+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.119+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.119+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.120+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.120+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.120+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.120+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.120+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.120+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.121+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.121+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.121+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.121+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.121+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.121+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.121+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.121+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.122+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.122+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.122+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.122+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.122+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.122+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.122+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.122+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.122+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.122+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.122+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.123+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.123+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.123+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.123+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.123+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.123+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.123+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.123+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.124+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.124+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.124+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.124+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.124+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.124+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.125+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.125+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.125+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.125+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.125+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.125+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.126+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.126+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.126+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.126+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.126+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.126+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.126+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.126+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.127+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.127+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.127+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.127+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.128+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.128+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.128+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.128+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.128+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.128+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.129+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.129+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.129+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.129+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.129+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.129+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.129+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.130+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.130+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.130+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.130+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.130+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.130+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.130+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.131+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.131+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.131+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.131+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.131+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.131+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.131+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.131+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.131+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.132+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.132+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.132+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.132+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.132+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.132+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.132+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.133+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.133+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.133+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.133+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.133+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.133+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.133+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.133+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.133+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.133+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.134+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.134+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.134+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.134+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.134+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.134+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.134+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.134+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.135+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.135+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.135+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.135+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.135+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.135+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.135+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.135+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.136+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.136+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.136+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.136+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.136+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.136+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.136+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.136+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.136+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.136+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.137+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.137+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.137+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.137+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.137+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.137+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.137+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.137+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.137+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.137+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.138+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.138+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.139+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.139+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.139+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.139+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.139+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.139+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.139+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.139+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.140+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.140+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.140+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.140+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.140+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.140+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.140+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.140+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.141+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.141+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.141+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.141+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.141+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.141+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.142+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.142+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.142+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.142+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.142+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.142+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.142+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.142+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.142+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.142+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.142+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.142+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.143+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.143+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.143+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.143+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.143+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.143+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.143+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.143+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.144+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.144+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.144+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.144+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.144+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.144+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.144+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.144+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.145+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.145+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.145+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.145+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.145+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.145+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.145+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.145+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.145+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.145+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.146+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.146+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.146+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.146+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.146+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.146+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.146+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.146+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.147+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.147+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.147+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.147+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.147+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.147+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.148+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.148+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.148+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.148+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.148+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.148+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.149+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.149+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.149+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.149+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.149+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.149+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.149+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.149+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.150+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.150+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.150+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.150+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.150+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.150+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.150+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.150+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.151+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.151+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.151+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.151+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.151+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.151+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.151+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.151+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.152+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.152+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.152+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.152+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.152+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.152+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.152+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.152+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.152+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.152+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.153+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.153+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.153+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.153+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.153+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.153+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.153+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.153+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.154+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.154+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.154+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.154+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.154+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.154+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.154+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.154+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.155+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.155+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.155+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.155+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.155+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.155+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.155+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.155+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.155+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.155+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.156+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.156+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.156+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.156+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.156+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.156+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.157+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.157+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.157+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.157+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.158+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.158+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.158+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.158+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.158+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.158+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.158+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.158+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.158+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.158+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.158+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.158+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.159+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.159+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.159+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.159+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.159+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.159+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.160+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.160+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.160+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.160+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.160+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.160+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.160+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.160+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.161+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.161+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.161+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.161+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.162+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.162+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.162+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.162+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.163+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.163+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.163+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.163+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.163+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.163+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.163+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.163+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.164+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.164+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.164+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.164+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.164+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.164+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.164+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.164+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.165+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.165+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.165+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.165+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.165+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.165+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.165+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.165+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.166+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.166+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.166+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.166+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.166+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.166+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.167+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.167+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.167+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.167+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.168+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.168+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.168+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.168+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.168+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.168+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.168+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.168+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.169+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.169+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.169+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.169+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.169+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.169+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.169+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.169+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.170+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.170+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.170+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.170+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.170+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.170+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.170+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.170+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.171+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.171+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.171+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.171+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.171+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.171+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.172+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.172+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.172+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.172+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.172+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.172+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.172+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.172+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.172+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.172+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.173+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.173+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.173+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.173+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.173+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.173+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.173+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.173+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.174+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.174+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.174+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.174+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.174+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.174+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.174+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.174+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.175+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.175+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.175+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.175+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.176+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.176+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.176+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.176+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.176+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.176+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.176+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.176+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.177+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.177+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.177+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.177+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.177+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.177+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.177+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.177+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.178+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.178+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.178+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.178+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.178+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.178+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.178+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.178+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.178+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.178+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.179+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.179+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.180+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.180+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.180+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.180+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.180+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.180+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.180+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.180+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.181+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.181+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.182+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.182+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.182+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.182+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.182+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.182+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.183+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.183+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.184+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.184+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.184+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.184+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.184+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.184+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.184+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.184+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.185+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.185+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.185+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.185+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.185+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.185+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.186+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.186+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.186+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.186+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.186+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.186+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.186+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.186+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.186+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.187+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.187+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.187+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.187+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.187+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.187+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.187+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.187+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.187+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.189+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.189+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.189+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.189+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.189+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.189+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.189+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.189+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.190+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.190+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.190+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.190+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.191+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.191+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.191+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.191+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.191+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.191+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.191+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.191+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.192+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.192+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.192+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.192+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.192+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.193+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.193+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.193+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.193+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.193+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.193+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.193+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.193+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.194+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.194+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.194+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.194+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.195+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.195+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.195+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.195+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.196+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.196+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.197+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.197+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.197+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.197+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.197+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.197+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.197+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.197+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.198+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.198+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.198+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.198+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.198+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.198+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.198+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.198+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.199+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.199+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.199+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.199+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.199+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.199+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.199+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.199+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.200+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.200+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.200+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.200+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.200+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.200+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.200+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.200+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.201+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.201+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.201+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.201+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.201+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.201+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.202+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.202+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.202+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.202+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.202+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.202+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.202+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.203+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.203+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.203+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.203+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.203+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.203+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.203+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.204+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.204+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.204+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.204+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.204+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.204+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.204+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.204+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.205+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.205+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.205+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.205+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.205+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.205+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.205+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.205+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.206+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.206+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.206+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.206+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.206+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.206+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.206+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.206+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.207+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.207+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.207+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.207+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.207+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.207+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.207+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.207+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.208+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.208+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.208+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.208+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.208+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.208+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.208+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.208+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.209+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.209+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.209+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.209+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.209+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.209+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.209+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.209+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.209+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.209+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.209+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.209+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.210+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.210+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.210+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.210+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.210+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.210+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.210+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.210+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.211+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.211+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.211+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.211+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.211+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.211+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.211+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.211+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.211+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.211+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.211+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.211+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.212+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.212+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.213+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.213+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.213+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.213+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.213+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.213+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.213+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.213+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.214+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.214+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.214+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.214+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.884 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:59.214+0000 7f3e5c637640 -1 rbd::mirror::LeaderWatcher: 0x560e1a3bd200 handle_notify_lock_released: error notifying leader lock released: (2) No such file or directory 2024-04-23T21:58:59.884 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:59.214+0000 7f3e5c637640 -1 rbd::mirror::LeaderWatcher: 0x560e1a3bd200 handle_shut_down_leader_lock: error shutting down leader lock: (2) No such file or directory 2024-04-23T21:58:59.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.884 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:59.214+0000 7f3e5be36640 -1 rbd::mirror::LeaderWatcher: 0x560e1a3bd200 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (2) No such file or directory 2024-04-23T21:58:59.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.214+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.214+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.214+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.214+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.885 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:59.215+0000 7f3e5be36640 -1 rbd::mirror::InstanceWatcher: 0x560e1a1b2a80 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.7861 object: (2) No such file or directory 2024-04-23T21:58:59.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.215+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.215+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.215+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.215+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.887 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:59.215+0000 7f3e4860f640 -1 rbd::mirror::InstanceWatcher: 0x560e1a1b2a80 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:58:59.887 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:59.215+0000 7f3e41e02640 -1 rbd::mirror::MirrorStatusUpdater 0x560e1a4426c0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:59.887 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:59.215+0000 7f3e5be36640 -1 rbd::mirror::NamespaceReplayer: 0x560e1a67a000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:58:59.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.216+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.216+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.216+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.216+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.889 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:59.216+0000 7f3e47e0e640 -1 rbd::mirror::MirrorStatusUpdater 0x560e1a4117a0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:59.889 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:59.216+0000 7f3e5c637640 -1 rbd::mirror::NamespaceReplayer: 0x560e1a67a000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:58:59.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.216+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.216+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.216+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.216+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.217+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.217+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.217+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.217+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.217+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.217+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.217+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.217+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.218+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.218+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.218+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.218+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.218+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.218+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.218+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.218+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.218+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.218+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.895 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:59.219+0000 7f3e5c637640 -1 rbd::mirror::LeaderWatcher: 0x560e1a118c00 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (2) No such file or directory 2024-04-23T21:58:59.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.219+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.219+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.219+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.219+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.896 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:59.219+0000 7f3e5be36640 -1 rbd::mirror::InstanceWatcher: 0x560e1a1b2000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.7858 object: (2) No such file or directory 2024-04-23T21:58:59.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.897 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:59.220+0000 7f3e55e2a640 -1 rbd::mirror::InstanceWatcher: 0x560e1a1b2000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:58:59.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.220+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.220+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.898 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:59.220+0000 7f3e4f61d640 -1 rbd::mirror::MirrorStatusUpdater 0x560e19f765a0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:59.898 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:59.220+0000 7f3e5c637640 -1 rbd::mirror::NamespaceReplayer: 0x560e1aa1e000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:58:59.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.220+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.220+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.220+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.899 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:59.220+0000 7f3e55629640 -1 rbd::mirror::MirrorStatusUpdater 0x560e19f13440 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:58:59.899 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi150.stderr:2024-04-23T21:58:59.220+0000 7f3e5c637640 -1 rbd::mirror::NamespaceReplayer: 0x560e1aa1e000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:58:59.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.220+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.220+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.221+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.221+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.221+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.221+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.221+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.221+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.222+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.222+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.222+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.222+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.222+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.222+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.222+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.222+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.222+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.222+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.223+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.223+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.223+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.223+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.223+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.223+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.224+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.224+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.224+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.224+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.224+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.224+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.225+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.225+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.225+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.225+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.225+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.225+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.225+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.225+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.226+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.226+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.226+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.226+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.227+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.227+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.227+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.227+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.227+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.227+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.228+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.228+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.228+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.228+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.228+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.228+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.228+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.228+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.229+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.229+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.229+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.229+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.229+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.229+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.230+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.230+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.230+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.230+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.230+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.230+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.230+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.230+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.231+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.231+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.231+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.231+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.231+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.231+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.231+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.231+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.232+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.232+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.232+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.232+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.232+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.232+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.233+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.233+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.233+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.233+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.233+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.233+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.233+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.233+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.234+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.234+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.234+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.234+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.234+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.234+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.235+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.235+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.235+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.235+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.235+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.235+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.235+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.235+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.236+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.236+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.236+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.236+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.236+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.236+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.236+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.236+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.236+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.236+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.237+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.237+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.237+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.237+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.237+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.237+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.237+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.237+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.238+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.238+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.238+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.238+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.238+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.238+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.238+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.238+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.239+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.239+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.239+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.239+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.239+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.239+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.240+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.240+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.240+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.240+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.240+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.240+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.240+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.240+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.241+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.241+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.241+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.241+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.241+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.241+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.241+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.241+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.242+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.242+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.242+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.242+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.242+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.242+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.242+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.242+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.243+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.243+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.243+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.243+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.243+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.243+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.243+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.243+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.243+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.243+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.244+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.244+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.244+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.244+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.244+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.244+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.245+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.245+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.246+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.246+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.246+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.246+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.246+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.246+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.247+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.247+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.247+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.247+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.247+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.247+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.247+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.247+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.248+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.248+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.248+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.248+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.248+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.248+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.248+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.248+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.249+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.249+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.249+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.249+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.249+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.249+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.249+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.249+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.250+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.250+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.250+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.250+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.250+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.250+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.251+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.251+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.251+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.251+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.252+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.252+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.252+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.252+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.253+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.253+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.253+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.253+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.253+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.253+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.253+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.253+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.254+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.254+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.254+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.254+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.254+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.254+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.254+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.254+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.255+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.255+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.255+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.255+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.255+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.255+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.255+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.255+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.255+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.255+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.256+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.256+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.256+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.256+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.256+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.256+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.256+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.256+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.256+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.256+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.257+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.257+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.257+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.257+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.257+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.257+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.257+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.258+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.258+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.258+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.258+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.258+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.258+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.258+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.259+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.259+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.259+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.259+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.259+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.259+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.260+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.260+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.260+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.260+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.260+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.260+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.260+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.260+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.261+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.261+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.261+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.261+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.261+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.261+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.261+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.261+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.262+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.262+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.262+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.262+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.262+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.262+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.262+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.262+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.263+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.263+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.263+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.263+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.263+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.263+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.263+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.263+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.264+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.264+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.265+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.265+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.265+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.265+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.265+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.265+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.265+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.265+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.266+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.266+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.266+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.266+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.266+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.266+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.266+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.266+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.267+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.267+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.267+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.267+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.267+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.267+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.267+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:58:59.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:58:59.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.269+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.269+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.269+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.269+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.270+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.270+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.270+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.270+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.271+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.271+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.271+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.271+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.272+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.272+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.272+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.272+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.273+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.273+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.273+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.273+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.274+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.274+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.274+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.274+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.276+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.276+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.276+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.276+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.277+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.277+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.277+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.277+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.278+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.278+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.278+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.278+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.279+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.279+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.279+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.279+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.280+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.280+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.280+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.280+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.281+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.281+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.281+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.281+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.282+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.282+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.282+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.282+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.283+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.283+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.283+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.283+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.284+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.284+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.284+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.284+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.285+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.285+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.285+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.285+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.286+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.286+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.286+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.286+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.287+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.287+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.287+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.287+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.288+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.288+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.269+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.269+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.286+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.286+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.288+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.288+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.267+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.268+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.268+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.268+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.268+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.269+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.269+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.269+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.269+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.270+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.270+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.270+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.270+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.271+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.271+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.271+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.271+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.272+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.272+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.272+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.272+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.273+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.273+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.274+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.274+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.274+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.274+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.275+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.275+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.275+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.275+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.276+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.276+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.276+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.276+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.277+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.277+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.277+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.277+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.278+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.278+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.278+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.278+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.279+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.279+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.279+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.279+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.280+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.280+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.281+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.281+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.282+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.282+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.282+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.282+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.283+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.283+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.283+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.283+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.284+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.284+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.284+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.284+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.285+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.285+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.285+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.285+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.286+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.286+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.286+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.286+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.287+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.287+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.288+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.288+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.288+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.288+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.288+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.288+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.288+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.288+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.289+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.289+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.289+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.289+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.289+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.289+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.290+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.290+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.292+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.292+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.292+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.292+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.292+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.292+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.292+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.292+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.293+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.293+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.293+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.293+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.293+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.293+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.293+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.293+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.293+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.294+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.294+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.294+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.294+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.294+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.294+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.294+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.294+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.294+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.294+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.294+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.295+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.295+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.295+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.295+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.295+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.295+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.295+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.295+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.295+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.295+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.295+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.295+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.296+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.296+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.296+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.296+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.296+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.296+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.296+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.296+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.296+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.296+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.296+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.296+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.297+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.297+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.297+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.297+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.297+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.297+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.297+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.297+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.297+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.297+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.297+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.297+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.298+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.298+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.298+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.298+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.298+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.298+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.298+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.298+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.299+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.299+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.299+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.299+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.299+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.299+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.300+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.300+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.300+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.300+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.300+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.300+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.300+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.300+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.301+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.301+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.301+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.301+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.301+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.301+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.301+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.301+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.301+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.301+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.302+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.302+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.302+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.302+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.302+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.302+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.302+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.302+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.302+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.302+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.302+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.302+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.303+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.303+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.303+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.303+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.303+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.303+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.303+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.303+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.304+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.304+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.304+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.304+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.304+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.304+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.305+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.305+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.305+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.305+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.305+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.305+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.305+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.305+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.306+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.306+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.306+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.306+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.306+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.306+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.306+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.306+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.307+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.307+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.307+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.307+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.307+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.307+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.307+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.307+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.308+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.308+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.308+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.308+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.308+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.308+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.308+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.308+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.308+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.308+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.309+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.309+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.309+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.309+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.309+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.309+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.310+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.310+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.310+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.310+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.310+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.310+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.310+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.310+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.311+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.311+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.311+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.311+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.311+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.311+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.311+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.311+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.311+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.311+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.312+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.312+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.312+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.312+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.312+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.312+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.312+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.312+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.313+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.313+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.313+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.313+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.313+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.313+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.313+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.313+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.313+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.313+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.314+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.314+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.314+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.314+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.314+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.314+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.314+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.314+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.314+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.314+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.315+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.315+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.315+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.315+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.315+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.315+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.315+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.315+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.316+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.316+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.316+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.316+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.316+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.316+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.316+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.316+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.316+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.316+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.317+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.317+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.317+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.317+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.317+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.317+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.317+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.317+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.317+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.317+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.317+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.317+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.318+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.318+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.318+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.318+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.318+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.318+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.318+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.318+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.318+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.318+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.319+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.319+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.319+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.319+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.319+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.320+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.320+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.320+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.320+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.321+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.321+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.321+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.321+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.322+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.322+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.322+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.322+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.323+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.323+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.323+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.323+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.324+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.324+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.324+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.324+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.325+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.325+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.325+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.325+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.326+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.326+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.326+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.326+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.327+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.327+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.327+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.327+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.328+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.328+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.328+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.328+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.329+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.329+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.329+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.329+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.330+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.330+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.331+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.331+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.331+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.331+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.332+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.332+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.332+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.332+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.333+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.333+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.333+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.333+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.334+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.334+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.334+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.334+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.335+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.335+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.335+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.335+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.336+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.336+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.336+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.336+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.337+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.337+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.338+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.338+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.338+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.338+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.339+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.339+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.339+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.339+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.340+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.340+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.340+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.340+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.341+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.341+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.341+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.341+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.342+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.342+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.343+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.343+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.343+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.343+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.344+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.344+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.344+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.344+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.345+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.345+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.345+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.345+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.346+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.346+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.346+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.346+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.347+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.347+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.348+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.348+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.348+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.348+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.349+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.349+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.349+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.349+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.344+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.344+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.345+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.345+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.345+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.345+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.348+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.348+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.348+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.348+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.319+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.320+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.320+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.320+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.320+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.321+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.321+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.321+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.321+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.322+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.322+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.322+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.322+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.323+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.323+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.323+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.323+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.324+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.324+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.324+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.324+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.325+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.325+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.325+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.325+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.326+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.326+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.326+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.326+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.326+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.326+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.327+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.327+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.327+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.327+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.328+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.328+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.328+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.328+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.329+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.329+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.329+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.329+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.330+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.330+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.330+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.330+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.331+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.331+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.331+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.331+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.332+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.332+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.332+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.332+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.333+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.333+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.333+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.333+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.335+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.335+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.335+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.335+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.337+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.337+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.338+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.338+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.339+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.339+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.339+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.339+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.340+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.340+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.340+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.340+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.341+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.341+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.341+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.341+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.342+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.342+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.342+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.342+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.343+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.343+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.343+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.343+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.344+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.344+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.344+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.344+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.345+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.345+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.346+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.346+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.346+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.346+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.347+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.347+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.347+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.347+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.348+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.348+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.348+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.348+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.349+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.349+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.349+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.349+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.350+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.350+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.350+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.350+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.350+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.350+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.350+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.350+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.351+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.351+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.351+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.351+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.351+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.351+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.351+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.351+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.352+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.352+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.352+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.352+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.352+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.352+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.352+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.352+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.353+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.353+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.353+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.353+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.353+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.353+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.353+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.354+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.354+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.354+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.354+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.354+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.354+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.354+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.355+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.355+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.355+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.355+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.355+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.355+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.355+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.355+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.356+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.356+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.356+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.356+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.356+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.356+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.356+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.356+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.357+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.357+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.357+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.357+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.357+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.357+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.357+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.357+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.358+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.358+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.358+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.358+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.358+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.358+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.358+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.358+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.359+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.359+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.359+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.359+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.359+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.359+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.359+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.359+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.360+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.360+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.361+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.361+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.361+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.361+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.361+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.361+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.361+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.361+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.362+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.362+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.362+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.362+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.362+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.362+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.362+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.362+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.363+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.363+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.363+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.363+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.363+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.363+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.363+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.363+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.364+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.364+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.364+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.364+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.364+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.364+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.364+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.364+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.364+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.364+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.364+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.364+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.365+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.365+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.365+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.365+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.365+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.365+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.366+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.366+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.366+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.366+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.367+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.367+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.367+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.367+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.367+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.367+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.367+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.367+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.368+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.368+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.368+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.368+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.368+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.368+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.368+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.368+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.369+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.369+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.369+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.369+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.369+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.369+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.370+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.370+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.370+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.370+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.370+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.370+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.370+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.370+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.371+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.371+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.371+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.371+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.371+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.371+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.371+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.371+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.372+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.372+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.372+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.372+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.372+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.372+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.372+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.372+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.373+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.373+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.373+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.373+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.373+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.373+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.373+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.373+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.374+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.374+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.374+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.374+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.374+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.374+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.374+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.374+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.375+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.375+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.375+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.375+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.375+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.375+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.375+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.375+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.376+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.376+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.376+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.376+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.376+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.376+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.376+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.376+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.377+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.377+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.377+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.377+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.377+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.377+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.377+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.377+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.378+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.378+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.378+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.378+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.378+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.378+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.379+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.379+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.379+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.379+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.380+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.380+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.380+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.380+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.381+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.381+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.381+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.381+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.382+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.382+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.382+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.382+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.383+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.383+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.383+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.383+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.384+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.384+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.384+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.384+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.385+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.385+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.385+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.385+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.386+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.386+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.386+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.386+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.387+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.387+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.387+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.387+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.388+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.388+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.388+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.388+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.389+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.389+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.389+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.389+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.390+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.390+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.390+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.390+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.391+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.391+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.391+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.391+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.392+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.392+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.392+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.392+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.393+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.393+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.393+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.393+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.394+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.394+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.394+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.394+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.395+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.395+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.395+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.395+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.396+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.396+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.396+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.396+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.397+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.397+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.397+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.397+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.398+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.398+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.398+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.398+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.399+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.399+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.400+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.400+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.400+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.400+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.401+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.401+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.401+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.401+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.402+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.402+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.402+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.402+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.403+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.403+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.403+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.403+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.404+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.404+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.404+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.404+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.405+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.405+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.405+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.405+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.406+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.406+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.406+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.406+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.407+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.407+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.408+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.408+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.408+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.408+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.409+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.409+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.409+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.409+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.410+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.410+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.410+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.410+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.411+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.411+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.411+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.411+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.412+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.412+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.412+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.412+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.413+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.413+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.413+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.413+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.414+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.414+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.414+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.414+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.415+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.415+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.416+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.416+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.416+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.416+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.417+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.417+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.417+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.417+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.418+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.418+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.418+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.418+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.419+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.419+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.419+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.419+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.420+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.420+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.421+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.421+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.421+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.421+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.422+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.422+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.422+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.422+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.423+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.423+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.423+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.423+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.424+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.424+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.424+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.424+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.425+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.425+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.425+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.425+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.426+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.426+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.427+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.427+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.427+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.427+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.428+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.428+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.389+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.389+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.389+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.389+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.394+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.394+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.395+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.395+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.407+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.407+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.408+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.408+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.408+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.418+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.418+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.418+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.418+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.419+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.419+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.425+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.425+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.379+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.379+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.379+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.379+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.380+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.380+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.380+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.380+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.381+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.381+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.382+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.382+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.382+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.382+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.383+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.383+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.383+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.383+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.384+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.384+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.384+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.384+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.385+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.385+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.385+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.385+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.386+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.386+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.386+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.386+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.387+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.387+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.387+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.387+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.388+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.388+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.388+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.388+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.389+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.389+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.389+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.389+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.391+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.391+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.391+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.391+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.392+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.392+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.392+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.392+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.393+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.393+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.393+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.393+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.394+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.394+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.394+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.394+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.395+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.395+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.395+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.395+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.396+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.396+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.396+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.396+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.397+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.397+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.397+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.397+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.398+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.398+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.398+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.398+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.399+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.399+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.399+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.399+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.400+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.400+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.400+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.400+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.401+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.401+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.403+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.403+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.403+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.403+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.404+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.404+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.404+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.404+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.405+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.405+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.405+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.405+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.406+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.406+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.406+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.406+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.407+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.407+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.407+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.407+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.408+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.408+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.408+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.408+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.409+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.409+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.409+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.409+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.410+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.410+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.410+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.410+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.411+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.411+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.411+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.411+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.412+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.412+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.412+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.412+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.413+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.413+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.413+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.413+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.414+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.414+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.414+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.414+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.415+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.415+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.415+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.415+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.416+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.416+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.416+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.416+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.418+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.418+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.418+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.418+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.419+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.419+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.419+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.419+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.420+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.420+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.420+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.420+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.421+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.421+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.422+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.422+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.422+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.422+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.423+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.423+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.423+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.423+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.424+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.424+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.424+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.424+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.425+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.425+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.425+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.425+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.426+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.426+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.426+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.426+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.428+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.428+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.429+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.429+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.430+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.430+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.430+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.430+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.431+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.431+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.431+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.431+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.432+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.432+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.433+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.433+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.433+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.433+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.434+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.434+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.434+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.434+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.435+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.435+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.435+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.435+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.436+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.436+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.436+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.436+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.437+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.437+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.437+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.437+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.438+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.438+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.438+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.438+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.439+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.439+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.440+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.440+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.440+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.440+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.441+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.441+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.441+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.441+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.442+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.442+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.442+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.442+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.443+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.443+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.443+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.443+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.444+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.444+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.444+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.444+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.445+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.445+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.445+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.445+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.446+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.446+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.446+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.446+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.447+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.447+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.447+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.447+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.448+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.448+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.448+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.448+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.449+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.449+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.449+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.449+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.450+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.450+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.450+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.450+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.451+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.451+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.451+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.451+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.452+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.452+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.452+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.452+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.453+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.453+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.454+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.454+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.454+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.454+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.455+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.455+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.455+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.455+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.456+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.456+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.456+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.456+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.457+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.457+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.457+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.457+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.458+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.458+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.458+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.458+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.459+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.459+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.459+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.459+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.460+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.460+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.460+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.460+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.461+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.461+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.461+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.461+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.462+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.462+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.462+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.462+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.463+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.463+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.463+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.463+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.464+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.464+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.465+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.465+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.465+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.465+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.466+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.466+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.466+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.466+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.467+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.467+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.467+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.467+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.468+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.468+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.468+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.468+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.469+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.469+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.469+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.469+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.470+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.470+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.470+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.470+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.471+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.471+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.471+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.471+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.472+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.472+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.472+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.472+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.473+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.473+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.473+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.473+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.474+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.474+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.474+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.475+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.475+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.475+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.476+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.476+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.476+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.476+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.477+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.477+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.477+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.477+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.478+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.478+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.436+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.436+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.439+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.439+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.445+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.445+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.446+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.446+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.447+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.447+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.451+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.451+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.454+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.454+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.464+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.464+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.427+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.427+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.428+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.428+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.428+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.428+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.429+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.429+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.429+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.429+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.430+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.430+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.430+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.430+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.432+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.432+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.432+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.432+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.433+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.433+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.433+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.433+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.434+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.434+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.434+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.434+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.436+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.436+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.436+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.436+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.437+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.437+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.437+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.437+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.438+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.438+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.438+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.438+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.439+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.439+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.439+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.439+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.440+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.440+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.440+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.440+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.442+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.442+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.442+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.442+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.444+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.444+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.444+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.444+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.445+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.445+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.445+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.445+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.446+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.446+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.446+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.446+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.447+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.447+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.447+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.447+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.448+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.448+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.448+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.448+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.449+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.449+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.449+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.449+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.450+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.450+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.450+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.450+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.451+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.451+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.451+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.451+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.451+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.452+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.452+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.452+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.452+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.452+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.453+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.453+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.453+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.453+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.454+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.454+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.454+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.454+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.456+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.456+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.456+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.456+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.457+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.457+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.457+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.457+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.458+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.458+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.458+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.458+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.459+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.459+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.459+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.459+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.460+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.460+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.460+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.460+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.461+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.461+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.461+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.461+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.462+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.462+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.462+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.462+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.463+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.463+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.463+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.463+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.464+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.464+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.464+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.464+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.465+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.465+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.465+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.465+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.466+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.466+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.466+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.466+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.467+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.467+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.467+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.467+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.468+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.468+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.468+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.468+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.469+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.469+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.470+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.470+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.470+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.470+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.471+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.471+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.471+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.471+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.472+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.472+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.472+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.472+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.473+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.473+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.478+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.478+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.479+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.479+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.479+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.479+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.480+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.480+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.480+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.480+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.481+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.481+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.481+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.481+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.482+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.482+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.482+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.482+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.483+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.483+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.484+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.484+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.484+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.484+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.485+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.485+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.485+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.485+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.486+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.486+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.486+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.486+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.487+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.487+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.487+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.487+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.488+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.488+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.488+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.488+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.489+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.489+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.489+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.489+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.490+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.490+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.490+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.491+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.491+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.491+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.492+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.492+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.492+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.492+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.493+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.493+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.493+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.493+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.494+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.494+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.494+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.494+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.495+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.495+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.495+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.495+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.496+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.496+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.496+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.496+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.497+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.497+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.497+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.497+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.498+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.498+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.499+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.499+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.499+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.499+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.501+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.501+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.502+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.502+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.503+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.503+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.503+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.503+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.504+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.504+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.504+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.504+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.505+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.505+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.505+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.505+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.506+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.506+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.506+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.506+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.507+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.507+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.508+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.508+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.508+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.508+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.509+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.509+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.509+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.509+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.510+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.510+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.510+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.510+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.511+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.511+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.511+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.511+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.512+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.512+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.512+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.512+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.513+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.513+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.515+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.515+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.516+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.516+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.516+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.516+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.517+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.517+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.517+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.517+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.518+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.518+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.518+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.518+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.519+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.519+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.520+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.520+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.520+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.520+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.521+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.521+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.521+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.521+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.522+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.522+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.522+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.522+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.523+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.523+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.523+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.523+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.524+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.524+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.524+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.524+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.525+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.525+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.525+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.525+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.526+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.526+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.526+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.526+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.527+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.527+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.527+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.527+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.528+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.528+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.528+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.528+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.529+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.529+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.530+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.530+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.530+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.530+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.531+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.531+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.531+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.531+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.532+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.532+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.491+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.491+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.492+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.492+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.501+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.501+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.502+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.502+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.513+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.513+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.524+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.524+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.530+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.530+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.530+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.530+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.531+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.531+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.473+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.473+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.474+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.474+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.474+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.474+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.475+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.475+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.475+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.475+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.476+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.476+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.476+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.476+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.477+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.477+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.477+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.477+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.478+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.478+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.478+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.478+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.479+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.479+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.479+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.479+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.480+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.480+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.480+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.480+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.481+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.481+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.481+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.481+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.482+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.482+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.482+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.482+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.482+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.482+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.483+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.483+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.484+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.484+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.484+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.484+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.485+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.485+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.485+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.485+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.486+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.486+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.488+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.488+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.489+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.489+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.489+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.489+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.490+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.490+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.490+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.490+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.491+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.491+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.491+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.491+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.492+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.492+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.492+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.492+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.493+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.493+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.493+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.493+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.494+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.494+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.494+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.494+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.495+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.495+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.495+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.495+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.496+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.496+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.496+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.496+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.497+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.497+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.497+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.497+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.498+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.498+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.498+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.498+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.499+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.499+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.499+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.499+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.500+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.500+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.500+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.500+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.501+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.501+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.501+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.501+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.502+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.502+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.502+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.502+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.503+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.503+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.503+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.503+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.504+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.504+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.504+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.504+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.505+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.505+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.505+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.505+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.506+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.506+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.506+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.506+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.506+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.506+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.507+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.507+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.507+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.507+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.508+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.508+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.508+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.509+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.509+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.509+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.510+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.510+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.510+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.510+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.511+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.511+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.511+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.511+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.512+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.512+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.512+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.512+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.513+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.513+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.513+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.513+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.514+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.514+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.514+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.514+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.515+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.515+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.515+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.515+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.517+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.517+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.518+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.518+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.518+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.518+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.519+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.519+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.519+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.519+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.520+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.520+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.520+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.520+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.521+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.521+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.521+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.521+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.522+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.522+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.522+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.522+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.523+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.523+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.532+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.532+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.533+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.533+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.533+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.533+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.534+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.534+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.534+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.534+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.535+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.535+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.535+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.535+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.536+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.536+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.536+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.536+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.537+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.537+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.537+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.537+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.538+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.538+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.538+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.538+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.539+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.539+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.539+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.539+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.540+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.540+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.540+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.540+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.541+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.541+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.541+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.541+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.542+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.542+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.542+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.542+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.543+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.543+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.543+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.543+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.544+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.544+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.545+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.545+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.545+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.545+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.546+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.546+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.546+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.546+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.547+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.547+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.547+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.547+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.548+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.548+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.548+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.548+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.549+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.549+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.549+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.549+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.550+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.550+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.550+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.550+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.551+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.551+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.551+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.551+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.552+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.552+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.552+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.552+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.553+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.553+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.553+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.553+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.554+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.554+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.554+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.554+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.555+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.555+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.555+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.555+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.556+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.556+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.556+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.557+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.557+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.557+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.558+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.558+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.558+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.558+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.559+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.559+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.559+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.559+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.560+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.560+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.560+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.560+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.561+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.561+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.561+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.561+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.562+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.562+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.562+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.562+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.563+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.563+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.563+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.563+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.564+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.564+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.564+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.564+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.567+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.567+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.567+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.567+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.568+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.568+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.568+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.568+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.569+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.569+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.569+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.569+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.570+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.570+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.570+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.570+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.571+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.571+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.571+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.571+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.572+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.572+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.572+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.572+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.573+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.573+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.573+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.573+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.574+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.574+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.574+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.574+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.575+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.575+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.576+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.576+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.576+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.576+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.577+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.577+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.577+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.577+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.578+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.578+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.578+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.578+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.579+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.579+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.579+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.579+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.580+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.580+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.580+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.580+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.581+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.581+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.581+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.581+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.582+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.582+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.583+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.583+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.583+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.541+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.541+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.541+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.541+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.548+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.548+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.548+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.548+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.555+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.555+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.555+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.555+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.567+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.567+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.572+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.572+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.523+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.523+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.524+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.524+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.524+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.524+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.525+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.525+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.525+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.525+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.526+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.526+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.526+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.526+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.527+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.527+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.527+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.527+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.528+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.528+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.528+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.528+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.529+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.529+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.529+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.529+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.530+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.530+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.530+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.530+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.531+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.531+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.531+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.531+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.532+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.532+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.532+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.532+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.533+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.533+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.533+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.533+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.534+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.534+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.534+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.534+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.535+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.535+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.535+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.535+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.536+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.536+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.536+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.536+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.536+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.536+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.537+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.537+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.537+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.537+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.538+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.538+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.538+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.538+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.539+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.539+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.539+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.539+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.540+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.540+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.540+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.540+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.541+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.541+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.541+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.541+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.542+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.542+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.542+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.542+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.543+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.543+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.543+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.543+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.544+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.544+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.544+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.544+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.545+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.545+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.545+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.545+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.546+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.546+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.546+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.546+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.547+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.547+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.547+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.547+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.548+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.548+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.548+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.548+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.549+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.549+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.549+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.549+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.550+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.550+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.550+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.550+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.551+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.551+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.551+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.551+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.552+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.552+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.552+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.552+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.553+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.553+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.553+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.553+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.553+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.554+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.554+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.554+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.554+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.554+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.555+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.555+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.555+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.555+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.556+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.556+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.556+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.556+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.557+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.557+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.557+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.557+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.558+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.558+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.558+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.558+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.559+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.559+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.559+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.559+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.560+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.560+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.560+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.560+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.561+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.561+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.561+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.561+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.562+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.562+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.562+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.562+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.563+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.563+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.563+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.563+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.564+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.564+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.564+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.564+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.565+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.565+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.566+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.566+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.566+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.566+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.567+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.567+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.567+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.567+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.567+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.567+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.568+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.568+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.568+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.568+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.569+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.569+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.569+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.583+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.584+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.584+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.584+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.584+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.585+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.585+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.585+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.585+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.586+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.586+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.586+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.586+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.587+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.587+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.587+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.587+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.588+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.588+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.588+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.588+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.589+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.589+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.589+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.589+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.590+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.590+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.590+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.590+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.591+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.591+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.591+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.591+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.592+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.592+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.592+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.592+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.593+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.593+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.593+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.593+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.594+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.594+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.594+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.594+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.595+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.595+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.595+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.595+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.596+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.596+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.596+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.596+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.597+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.597+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.597+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.597+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.598+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.598+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.598+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.598+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.599+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.599+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.599+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.599+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.600+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.600+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.600+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.600+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.601+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.601+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.601+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.601+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.602+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.602+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.604+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.604+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.604+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.604+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.605+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.605+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.605+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.605+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.606+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.606+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.606+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.606+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.607+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.607+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.607+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.607+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.608+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.608+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.608+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.608+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.609+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.609+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.609+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.609+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.610+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.610+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.610+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.610+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.611+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.611+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.611+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.611+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.612+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.612+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.612+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.612+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.612+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.612+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.613+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.613+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.613+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.613+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.614+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.614+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.614+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.614+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.615+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.615+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.615+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.615+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.616+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.616+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.616+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.616+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.617+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.617+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.617+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.617+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.618+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.618+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.618+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.618+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.619+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.619+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.620+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.620+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.620+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.620+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.621+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.621+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.621+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.621+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.622+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.622+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.622+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.622+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.623+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.623+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.623+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.623+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.624+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.624+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.624+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.624+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.625+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.625+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.625+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.625+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.626+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.626+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.626+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.626+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.626+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.626+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.627+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.627+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.627+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.627+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.628+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.628+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.628+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.628+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.629+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.629+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.629+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.629+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.630+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.630+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.630+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.630+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.631+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.631+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.631+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.585+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.585+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.585+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.598+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.598+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.598+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.598+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.602+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.602+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.602+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.618+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.618+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.623+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.623+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.569+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.570+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.570+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.570+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.570+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.571+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.571+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.571+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.571+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.572+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.572+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.572+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.572+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.573+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.573+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.573+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.573+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.574+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.574+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.574+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.574+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.575+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.575+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.575+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.575+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.576+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.576+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.576+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.576+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.577+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.577+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.578+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.578+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.578+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.578+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.579+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.579+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.579+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.579+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.580+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.580+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.580+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.580+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.581+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.581+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.581+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.581+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.582+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.582+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.582+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.582+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.583+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.583+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.583+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.583+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.584+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.584+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.584+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.584+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.585+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.585+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.585+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.585+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.586+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.586+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.586+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.586+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.587+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.587+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.587+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.587+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.588+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.588+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.588+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.588+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.589+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.589+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.590+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.590+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.590+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.590+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.590+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.590+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.591+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.591+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.591+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.591+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.592+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.592+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.592+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.592+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.593+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.593+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.593+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.593+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.594+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.594+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.594+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.594+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.595+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.595+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.595+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.595+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.596+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.596+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.596+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.596+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.597+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.597+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.597+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.597+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.598+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.598+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.598+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.598+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.599+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.599+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.599+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.599+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.600+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.600+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.600+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.600+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.600+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.600+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.601+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.601+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.601+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.601+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.602+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.602+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.602+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.602+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.603+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.603+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.603+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.603+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.604+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.604+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.604+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.604+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.605+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.605+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.605+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.605+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.606+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.606+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.606+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.606+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.607+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.607+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.607+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.607+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.608+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.608+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.608+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.608+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.609+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.609+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.609+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.609+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.610+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.610+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.610+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.610+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.611+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.611+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.611+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.611+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.612+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.612+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.612+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.612+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.612+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.612+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.613+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.613+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.613+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.613+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.614+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.614+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.614+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.614+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.615+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.615+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.615+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.615+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.616+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.631+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.633+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.633+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.633+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.633+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.634+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.634+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.634+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.634+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.635+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.635+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.635+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.635+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.636+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.636+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.636+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.636+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.637+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.637+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.637+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.637+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.638+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.638+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.638+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.638+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.639+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.639+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.639+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.639+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.640+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.640+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.641+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.641+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.641+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.641+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.642+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.642+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.642+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.642+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.643+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.643+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.643+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.643+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.644+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.644+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.644+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.644+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.645+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.645+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.645+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.645+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.646+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.646+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.646+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.646+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.647+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.647+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.647+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.647+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.648+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.648+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.648+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.648+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.649+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.649+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.649+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.649+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.650+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.650+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.650+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.650+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.651+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.651+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.651+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.651+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.652+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.652+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.652+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.652+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.653+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.653+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.653+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.653+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.654+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.654+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.654+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.654+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.655+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.655+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.655+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.655+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.656+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.656+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.656+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.656+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.657+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.657+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.657+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.657+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.658+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.658+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.658+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.658+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.659+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.659+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.659+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.659+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.660+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.660+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.660+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.660+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.661+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.661+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.661+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.661+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.662+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.662+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.662+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.662+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.663+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.663+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.663+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.663+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.664+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.664+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.664+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.664+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.665+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.665+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.665+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.665+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.667+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.667+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.668+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.668+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.668+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.668+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.669+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.669+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.669+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.669+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.670+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.670+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.670+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.670+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.671+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.671+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.672+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.672+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.672+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.672+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.673+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.673+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.673+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.673+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.674+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.674+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.674+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.674+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.675+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.675+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.675+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.675+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.676+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.676+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.676+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.676+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.677+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.677+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.677+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.677+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.678+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.678+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.679+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.679+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.679+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.679+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.680+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.680+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.681+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.640+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.640+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.642+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.642+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.642+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.616+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.616+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.616+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.617+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.617+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.617+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.617+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.618+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.618+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.618+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.618+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.619+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.619+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.619+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.619+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.620+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.620+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.620+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.620+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.621+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.621+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.621+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.621+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.622+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.622+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.622+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.622+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.623+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.623+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.623+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.623+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.624+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.624+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.624+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.624+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.625+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.625+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.625+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.625+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.626+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.626+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.626+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.626+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.627+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.627+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.627+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.627+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.628+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.628+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.628+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.628+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.629+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.629+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.629+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.629+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.629+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.629+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.630+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.630+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.630+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.630+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.631+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.631+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.631+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.631+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.632+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.632+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.632+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.632+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.633+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.633+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.633+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.633+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.634+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.634+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.634+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.634+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.635+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.635+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.635+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.635+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.636+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.636+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.636+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.636+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.637+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.637+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.637+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.637+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.638+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.638+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.638+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.638+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.638+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.638+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.639+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.639+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.639+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.639+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.640+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.640+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.640+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.640+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.641+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.641+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.641+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.641+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.642+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.642+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.642+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.642+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.643+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.643+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.643+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.643+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.644+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.644+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.644+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.644+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.645+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.645+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.645+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.645+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.645+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.645+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.646+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.646+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.646+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.646+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.647+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.647+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.647+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.647+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.648+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.648+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.648+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.648+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.649+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.649+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.649+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.649+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.650+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.650+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.650+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.650+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.651+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.651+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.651+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.651+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.652+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.652+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.652+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.652+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.654+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.654+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.654+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.654+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.655+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.655+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.655+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.655+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.656+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.656+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.656+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.656+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.657+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.657+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.657+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.657+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.658+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.658+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.658+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.658+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.659+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.659+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.661+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.661+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.661+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.661+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.662+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.662+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.662+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.681+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.681+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.681+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.682+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.682+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.682+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.682+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.683+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.683+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.683+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.683+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.685+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.685+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.685+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.685+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.686+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.686+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.686+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.686+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.687+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.687+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.687+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.687+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.689+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.689+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.689+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.689+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.690+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.690+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.690+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.690+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.692+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.692+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.692+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.692+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.694+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.694+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.695+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.695+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.695+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.695+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.698+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.698+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.698+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.698+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.699+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.699+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.699+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.699+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.700+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.700+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.700+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.700+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.701+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.701+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.702+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.702+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.702+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.702+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.703+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.703+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.703+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.703+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.704+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.704+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.704+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.704+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.706+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.706+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.706+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.706+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.707+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.707+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.707+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.707+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.708+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.708+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.708+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.708+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.709+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.709+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.709+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.709+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.710+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.710+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.710+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.710+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.711+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.711+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.711+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.711+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.712+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.712+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.712+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.712+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.713+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.713+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.713+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.713+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.714+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.714+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.714+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.714+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.715+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.715+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.715+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.715+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.716+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.716+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.716+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.716+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.717+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.717+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.717+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.717+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.719+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.719+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.719+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.719+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.720+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.720+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.720+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.720+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.721+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.721+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.721+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.721+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.722+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.722+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.722+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.722+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.723+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.723+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.723+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.723+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.724+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.724+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.724+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.724+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.725+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.725+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.725+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.725+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.726+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.726+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.726+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.726+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.727+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.727+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.727+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.727+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.728+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.728+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.728+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.728+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.729+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.729+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.729+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.729+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.730+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.730+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.686+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.686+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.688+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.688+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.688+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.688+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.689+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.689+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.703+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.703+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.703+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.703+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.704+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.704+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.704+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.704+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.705+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.705+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.705+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.705+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.708+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.708+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.716+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.716+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.716+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.716+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.718+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.718+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.719+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.719+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.662+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.663+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.663+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.663+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.663+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.664+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.664+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.664+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.664+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.665+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.665+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.665+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.665+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.665+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.666+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.666+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.666+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.666+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.666+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.667+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.667+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.668+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.668+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.668+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.668+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.668+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.668+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.669+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.669+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.669+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.669+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.670+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.670+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.670+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.670+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.671+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.671+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.671+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.671+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.672+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.672+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.672+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.673+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.673+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.673+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.674+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.674+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.674+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.674+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.674+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.674+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.675+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.675+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.675+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.675+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.676+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.676+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.676+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.676+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.677+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.677+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.677+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.677+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.678+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.678+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.678+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.678+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.679+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.679+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.679+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.679+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.680+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.680+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.680+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.680+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.681+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.681+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.681+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.681+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.682+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.682+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.682+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.682+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.683+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.683+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.683+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.683+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.683+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.683+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.684+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.684+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.684+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.685+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.685+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.685+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.685+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.685+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.686+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.686+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.686+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.686+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.687+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.687+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.687+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.687+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.688+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.688+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.688+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.688+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.689+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.689+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.689+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.689+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.690+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.690+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.690+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.690+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.691+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.691+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.691+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.691+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.692+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.692+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.692+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.692+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.693+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.693+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.693+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.693+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.694+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.694+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.694+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.694+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.694+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.694+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.695+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.695+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.695+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.695+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.696+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.696+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.696+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.696+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.697+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.697+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.697+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.697+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.698+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.698+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.698+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.698+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.699+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.699+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.699+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.699+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.700+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.700+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.700+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.700+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.701+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.701+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.701+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.701+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.702+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.702+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.702+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.702+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.703+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.703+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.703+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.703+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.704+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.704+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.704+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.704+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.705+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.705+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.705+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.705+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.706+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.706+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.706+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.706+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.707+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.707+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.707+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.707+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.708+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.708+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.730+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.730+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.731+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.731+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.731+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.731+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.732+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.732+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.732+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.732+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.733+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.733+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.733+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.733+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.734+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.734+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.734+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.734+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.735+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.735+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.735+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.735+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.736+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.736+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.736+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.736+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.737+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.737+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.737+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.737+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.738+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.738+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.738+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.738+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.739+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.739+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.739+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.739+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.740+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.740+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.740+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.740+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.741+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.741+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.741+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.741+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.742+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.742+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.743+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.743+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.743+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.743+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.744+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.744+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.744+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.744+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.745+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.745+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.745+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.745+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.746+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.746+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.746+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.746+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.747+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.747+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.747+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.747+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.749+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.749+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.749+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.749+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.750+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.750+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.750+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.750+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.751+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.751+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.752+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.752+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.752+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.752+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.753+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.753+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.753+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.753+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.754+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.754+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.754+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.754+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.755+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.755+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.755+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.755+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.757+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.757+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.758+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.758+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.759+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.759+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.759+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.759+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.760+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.760+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.760+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.760+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.761+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.761+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.761+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.761+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.762+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.762+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.762+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.762+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.763+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.763+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.763+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.763+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.764+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.764+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.764+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.764+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.765+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.765+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.765+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.765+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.766+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.766+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.766+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.766+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.767+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.767+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.768+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.768+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.768+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.768+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.769+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.769+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.769+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.769+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.770+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.770+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.770+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.770+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.772+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.772+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.772+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.772+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.773+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.773+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.773+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.773+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.774+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.774+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.774+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.774+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.775+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.775+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.775+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.775+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.776+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.776+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.776+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.776+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.777+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.777+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.777+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.777+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.778+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.778+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.778+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.778+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.780+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.780+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.780+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.780+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.781+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.781+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.729+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.729+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.734+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.734+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.734+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.734+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.736+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.736+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.751+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.751+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.754+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.756+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.756+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.769+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.769+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.769+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.769+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.770+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.770+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.771+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.771+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.772+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.772+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.778+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.778+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.708+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.708+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.709+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.709+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.709+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.709+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.709+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.709+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.710+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.710+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.710+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.710+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.711+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.711+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.711+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.711+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.712+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.712+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.712+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.712+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.713+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.713+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.713+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.713+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.715+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.715+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.715+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.715+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.716+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.716+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.716+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.716+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.717+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.717+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.717+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.717+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.717+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.717+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.718+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.718+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.718+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.718+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.719+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.719+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.719+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.719+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.720+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.720+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.720+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.720+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.721+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.721+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.721+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.721+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.722+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.722+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.722+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.722+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.723+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.723+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.723+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.723+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.724+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.724+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.724+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.724+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.725+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.725+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.725+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.725+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.725+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.725+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.726+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.726+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.726+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.726+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.727+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.727+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.727+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.727+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.728+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.728+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.728+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.728+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.730+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.730+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.730+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.730+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.731+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.731+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.731+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.731+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.731+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.731+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.732+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.732+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.732+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.732+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.733+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.733+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.733+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.733+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.734+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.734+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.734+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.734+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.735+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.735+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.735+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.735+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.737+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.737+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.737+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.737+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.738+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.738+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.738+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.738+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.739+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.739+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.739+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.739+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.740+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.740+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.740+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.740+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.740+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.740+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.741+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.741+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.741+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.741+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.742+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.742+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.743+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.743+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.743+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.743+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.744+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.744+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.744+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.744+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.745+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.745+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.745+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.745+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.746+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.746+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.746+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.746+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.747+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.747+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.747+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.747+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.748+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.748+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.748+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.748+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.749+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.749+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.750+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.750+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.751+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.751+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.751+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.752+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.752+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.753+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.753+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.753+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.753+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.753+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.753+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.754+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.754+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.781+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.781+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.782+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.782+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.782+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.782+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.783+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.783+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.783+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.783+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.784+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.784+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.784+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.784+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.785+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.785+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.785+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.785+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.786+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.786+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.786+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.786+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.787+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.787+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.787+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.787+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.788+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.788+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.788+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.788+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.788+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.789+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.789+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.789+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.790+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.790+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.790+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.790+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.791+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.791+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.791+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.791+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.793+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.793+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.793+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.793+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.794+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.794+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.794+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.794+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.795+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.795+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.795+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.795+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.796+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.796+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.796+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.796+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.797+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.797+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.797+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.797+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.798+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.798+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.798+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.798+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.799+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.799+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.799+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.799+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.800+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.801+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.801+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.801+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.801+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.802+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.802+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.802+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.802+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.803+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.803+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.803+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.803+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.804+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.804+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.804+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.804+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.805+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.805+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.805+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.805+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.806+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.806+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.806+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.806+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.807+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.808+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.808+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.808+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.808+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.809+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.809+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.809+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.809+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.810+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.810+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.810+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.810+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.811+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.811+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.811+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.811+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.812+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.812+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.812+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.812+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.813+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.813+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.813+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.813+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.814+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.814+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.814+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.814+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.815+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.815+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.815+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.815+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.816+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.816+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.816+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.816+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.817+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.817+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.817+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.817+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.818+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.818+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.818+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.818+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.819+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.819+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.819+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.819+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.820+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.820+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.820+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.820+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.821+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.821+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.821+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.821+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.822+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.822+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.822+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.822+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.823+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.823+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.823+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.823+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.824+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.824+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.824+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.824+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.825+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.825+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.825+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.825+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.826+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.826+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.826+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.826+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.827+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.827+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.827+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.827+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.828+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.828+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.778+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.778+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.780+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.780+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.780+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.780+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.790+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.790+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.790+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.790+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.791+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.791+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.793+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.793+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.795+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.795+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.795+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.795+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.796+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.796+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.797+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.797+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.797+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.797+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.801+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.801+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.801+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.801+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.808+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.808+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.808+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.808+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.755+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.755+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.755+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.755+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.756+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.756+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.756+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.756+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.758+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.758+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.759+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.759+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.759+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.759+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.760+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.760+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.760+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.760+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.761+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.761+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.761+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.761+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.762+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.762+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.762+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.762+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.763+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.763+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.763+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.763+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.764+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.764+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.764+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.764+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.765+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.765+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.765+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.765+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.766+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.766+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.766+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.766+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.767+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.767+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.767+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.767+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.768+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.768+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.768+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.768+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.769+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.769+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.769+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.769+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.770+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.770+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.770+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.770+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.771+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.771+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.771+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.771+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.772+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.772+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.772+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.772+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.773+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.773+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.773+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.773+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.774+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.774+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.774+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.774+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.775+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.775+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.775+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.775+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.776+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.776+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.776+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.776+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.777+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.777+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.777+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.777+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.778+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.778+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.778+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.778+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.779+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.779+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.779+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.779+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.779+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.780+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.780+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.780+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.780+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.780+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.782+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.782+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.782+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.782+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.783+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.783+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.783+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.783+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.784+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.784+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.784+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.784+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.785+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.785+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.785+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.785+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.786+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.786+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.786+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.786+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.787+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.787+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.787+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.787+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.787+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.787+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.788+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.788+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.788+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.788+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.789+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.789+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.789+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.789+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.790+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.790+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.790+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.790+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.792+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.792+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.793+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.793+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.793+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.793+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.794+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.794+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.794+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.794+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.795+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.795+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.795+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.795+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.796+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.796+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.796+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.796+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.797+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.797+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.797+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.797+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.798+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.798+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.798+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.798+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.798+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.799+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.799+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.799+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.799+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.799+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.800+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.800+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.800+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.800+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.801+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.801+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.801+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.801+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.802+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.802+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.828+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.828+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.829+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.829+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.829+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.829+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.830+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.830+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.830+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.830+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.832+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.832+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.832+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.832+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.833+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.833+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.833+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.833+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.834+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.834+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.834+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.834+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.835+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.835+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.835+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.835+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.836+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.836+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.836+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.836+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.837+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.837+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.837+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.837+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.838+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.838+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.838+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.838+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.839+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.839+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.839+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.839+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.840+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.840+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.842+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.842+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.842+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.842+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.843+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.843+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.843+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.843+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.844+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.844+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.844+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.844+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.846+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.846+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.846+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.846+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.847+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.847+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.847+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.847+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.848+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.848+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.848+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.848+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.849+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.849+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.849+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.849+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.850+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.850+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.850+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.850+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.851+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.851+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.851+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.851+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.852+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.852+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.852+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.852+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.853+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.853+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.853+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.853+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.854+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.854+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.854+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.854+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.855+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.855+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.855+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.855+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.856+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.856+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.856+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.856+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.857+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.857+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.857+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.857+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.857+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.857+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.858+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.858+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.858+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.858+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.859+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.859+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.859+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.859+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.860+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.860+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.860+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.860+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.861+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.861+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.861+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.861+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.862+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.862+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.862+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.862+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.863+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.863+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.863+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.863+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.864+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.864+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.864+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.864+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.865+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.865+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.865+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.865+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.866+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.866+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.866+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.866+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.867+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.867+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.867+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.867+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.868+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.868+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.868+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.868+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.869+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.869+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.869+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.869+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.870+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.870+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.870+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.870+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.871+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.871+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.871+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.871+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.871+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.871+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.872+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.872+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.872+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.872+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.873+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.873+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.873+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.873+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.874+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.874+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.874+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.874+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.875+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.834+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.834+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.834+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.834+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.858+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.858+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.858+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.858+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.859+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.859+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.859+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.859+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.870+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.870+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.802+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.802+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.803+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.803+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.803+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.803+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.804+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.804+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.804+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.804+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.805+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.805+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.805+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.805+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.806+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.806+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.806+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.806+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.808+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.808+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.808+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.808+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.809+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.809+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.809+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.809+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.811+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.811+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.811+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.811+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.812+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.812+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.812+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.812+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.813+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.813+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.813+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.813+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.814+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.814+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.815+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.815+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.815+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.815+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.816+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.816+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.816+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.816+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.817+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.817+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.817+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.817+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.819+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.819+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.819+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.819+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.820+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.820+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.820+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.820+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.820+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.820+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.821+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.821+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.821+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.821+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.822+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.822+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.822+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.822+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.823+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.823+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.823+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.823+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.824+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.824+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.825+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.825+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.825+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.825+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.825+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.825+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.826+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.826+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.826+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.826+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.827+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.827+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.827+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.827+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.828+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.828+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.828+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.828+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.829+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.829+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.829+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.829+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.830+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.830+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.830+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.830+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.831+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.831+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.831+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.831+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.832+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.832+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.832+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.832+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.833+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.833+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.833+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.833+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.834+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.834+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.834+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.834+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.835+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.835+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.835+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.835+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.835+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.836+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.836+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.836+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.836+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.836+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.838+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.838+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.838+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.838+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.839+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.839+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.839+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.839+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.840+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.840+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.840+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.840+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.841+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.841+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.841+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.841+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.842+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.842+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.842+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.842+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.843+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.843+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.843+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.843+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.844+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.844+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.844+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.844+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.845+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.845+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.845+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.845+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.846+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.846+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.846+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.846+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.847+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.847+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.847+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.847+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.848+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.848+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.848+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.848+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.849+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.875+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.875+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.875+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.876+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.876+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.876+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.876+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.877+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.877+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.877+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.877+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.878+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.878+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.878+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.878+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.879+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.879+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.879+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.879+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.880+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.880+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.880+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.880+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.881+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.881+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.881+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.881+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.882+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.882+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.882+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.882+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.883+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.883+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.883+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.883+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.884+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.884+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.884+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.884+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.885+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.885+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.885+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.885+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.886+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.886+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.886+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.886+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.887+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.887+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.887+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.887+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.888+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.888+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.888+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.888+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.889+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.889+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.889+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.889+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.890+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.890+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.890+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.890+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.891+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.891+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.891+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.891+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.891+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.891+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.892+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.892+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.892+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.892+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.893+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.893+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.893+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.893+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.894+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.894+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.895+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.895+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.895+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.895+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.896+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.896+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.896+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.896+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.897+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.897+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.897+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.897+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.898+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.898+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.898+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.898+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.899+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.899+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.899+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.899+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.900+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.900+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.900+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.900+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.901+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.901+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.901+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.901+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.902+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.902+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.902+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.902+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.903+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.903+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.903+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.903+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.904+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.904+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.904+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.904+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.905+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.905+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.905+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.905+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.906+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.906+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.906+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.906+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.907+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.907+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.907+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.907+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.908+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.908+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.908+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.908+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.909+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.909+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.909+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.909+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.910+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.910+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.910+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.910+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.911+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.911+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.911+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.911+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.913+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.913+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.913+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.913+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.914+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.914+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.914+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.914+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.916+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.916+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.916+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.916+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.917+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.917+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.917+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.917+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.918+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.918+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.918+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.918+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.919+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.919+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.919+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.919+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.920+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.920+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.920+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.920+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.921+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.921+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.921+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.885+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.885+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.885+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.885+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.888+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.888+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.888+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.888+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.889+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.889+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.889+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.889+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.894+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.896+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.896+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.849+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.849+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.849+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.850+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.850+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.850+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.850+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.851+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.851+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.851+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.851+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.852+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.852+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.852+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.852+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.853+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.853+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.853+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.853+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.854+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.854+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.854+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.854+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.855+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.855+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.855+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.855+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.856+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.856+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.856+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.856+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.857+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.857+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.857+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.857+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.859+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.859+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.859+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.859+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.860+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.860+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.860+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.860+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.861+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.861+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.861+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.861+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.862+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.862+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.862+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.862+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.863+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.863+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.863+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.863+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.864+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.864+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.864+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.864+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.865+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.865+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.865+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.865+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.867+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.867+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.867+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.867+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.868+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.868+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.868+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.868+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.869+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.869+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.869+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.869+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.870+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.870+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.870+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.870+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.871+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.871+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.871+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.871+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.872+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.872+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.872+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.872+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.873+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.873+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.873+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.873+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.874+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.874+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.874+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.874+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.875+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.875+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.875+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.875+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.876+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.876+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.876+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.876+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.876+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.876+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.877+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.877+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.877+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.877+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.878+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.878+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.878+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.878+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.879+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.879+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.879+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.879+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.880+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.880+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.880+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.880+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.881+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.881+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.881+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.881+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.883+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.883+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.883+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.883+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.884+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.884+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.885+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.885+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.885+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.885+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.886+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.886+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.886+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.886+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.887+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.887+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.887+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.887+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.888+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.888+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.888+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.888+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.888+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.888+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.889+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.889+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.889+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.889+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.890+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.890+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.890+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.890+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.891+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.891+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.891+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.891+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.892+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.892+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.892+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.892+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.893+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.893+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.893+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.893+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.894+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.894+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.894+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.921+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.922+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.922+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.922+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.922+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.923+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.923+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.923+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.923+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.924+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.924+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.924+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.924+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.925+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.925+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.925+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.926+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.926+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.926+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.926+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.927+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.927+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.927+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.927+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.927+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.928+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.928+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.928+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.928+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.929+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.929+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.930+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.930+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.930+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.930+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.932+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.932+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.932+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.932+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.932+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.933+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.933+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.933+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.933+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.933+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.934+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.934+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.934+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.934+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.935+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.935+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.935+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.935+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.936+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.936+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.936+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.936+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.937+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.937+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.937+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.937+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.938+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.938+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.938+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.938+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.939+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.939+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.939+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.939+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.940+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.940+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.940+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.940+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.941+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.941+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.941+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.941+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.942+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.942+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.942+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.942+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.943+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.943+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.944+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.944+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.944+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.944+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.945+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.945+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.945+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.945+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.946+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.946+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.946+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.946+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.947+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.947+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.947+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.947+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.948+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.948+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.949+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.949+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.949+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.949+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.950+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.950+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.950+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.950+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.951+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.951+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.951+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.951+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.952+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.952+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.952+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.952+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.953+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.953+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.953+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.953+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.954+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.954+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.956+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.956+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.956+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.956+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.957+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.957+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.957+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.957+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.958+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.958+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.958+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.958+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.959+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.959+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.959+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.959+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.960+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.960+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.960+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.960+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.961+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.961+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.961+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.961+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.962+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.962+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.963+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.963+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.963+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.963+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.964+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.964+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.964+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.964+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.965+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.965+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.965+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.965+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.966+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.966+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.966+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.966+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.967+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.967+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.968+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.968+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.968+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.968+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.969+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.969+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.969+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.969+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.970+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.970+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.970+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.970+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.971+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.926+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.926+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.935+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.935+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.936+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.936+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.936+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.936+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.941+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.948+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.948+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.950+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.950+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.950+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.950+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.951+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.951+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.953+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.953+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.955+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.955+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.967+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.967+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.968+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.968+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.968+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.968+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.969+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.894+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.895+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.895+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.895+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.895+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.896+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.896+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.896+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.896+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.897+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.897+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.897+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.897+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.899+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.899+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.900+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.900+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.900+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.900+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.901+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.901+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.901+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.901+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.902+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.902+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.902+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.902+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.903+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.903+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.903+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.903+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.904+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.904+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.904+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.904+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.904+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.904+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.905+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.905+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.905+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.905+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.906+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.906+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.906+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.906+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.907+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.907+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.907+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.907+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.908+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.908+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.908+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.908+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.909+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.909+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.909+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.909+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.910+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.910+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.910+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.910+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.911+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.911+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.911+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.911+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.912+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.912+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.912+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.912+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.913+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.913+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.913+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.913+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.914+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.914+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.914+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.914+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.915+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.915+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.915+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.915+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.916+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.916+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.916+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.916+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.916+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.917+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.917+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.917+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.917+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.917+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.918+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.918+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.918+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.918+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.919+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.919+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.920+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.920+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.920+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.920+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.921+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.921+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.921+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.921+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.922+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.922+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.922+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.922+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.923+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.923+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.923+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.923+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.924+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.924+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.924+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.924+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.925+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.925+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.925+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.925+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.926+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.926+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.926+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.926+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.927+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.927+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.927+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.927+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.928+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.928+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.928+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.928+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.929+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.929+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.929+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.929+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.930+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.930+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.930+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.930+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.931+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.931+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.931+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.931+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.932+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.932+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.932+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.932+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.933+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.933+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.934+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.934+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.934+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.934+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.935+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.935+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.935+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.935+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.936+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.936+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.936+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.936+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.937+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.937+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.937+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.937+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.938+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.938+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.938+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.938+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.939+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.939+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.939+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.939+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.940+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.940+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.940+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.940+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.941+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.941+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.941+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.941+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.942+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.942+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.942+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.942+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.943+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.971+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.971+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.971+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.972+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.972+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.972+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.972+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.973+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.973+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.973+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.973+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.974+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.974+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.974+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.974+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.975+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.975+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.975+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.975+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.976+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.976+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.977+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.977+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.977+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.977+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.978+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.978+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.978+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.978+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.979+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.979+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.979+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.979+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.980+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.980+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.980+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.980+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.981+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.981+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.981+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.981+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.982+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.982+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.982+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.982+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.984+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.984+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.984+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.984+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.985+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.985+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.986+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.986+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.988+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.988+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.988+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.988+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.989+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.989+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.990+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.990+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.990+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.990+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.991+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.991+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.991+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.991+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.992+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.992+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.992+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.992+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.993+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.993+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.993+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.993+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.994+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.994+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.994+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.994+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.995+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.995+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.995+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.995+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.996+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.996+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.996+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.996+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.997+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.997+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.997+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.997+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.998+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.998+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.998+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.998+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.999+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:58:59.999+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.000+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.000+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.000+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.000+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.001+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.001+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.001+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.001+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.002+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.002+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.002+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.002+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.003+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.003+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.003+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.003+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.004+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.004+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.005+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.005+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.005+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.005+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.006+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.006+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.006+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.006+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.007+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.007+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.007+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.007+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.008+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.008+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.008+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.008+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.009+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.009+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.009+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.009+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.010+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.010+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.011+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.011+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.011+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.011+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.012+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.012+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.012+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.012+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.014+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.014+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.014+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.014+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.015+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.015+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.015+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.015+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.016+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.016+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.016+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.016+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.018+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.018+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.018+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.018+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.020+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.020+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.020+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.020+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.021+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.021+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.022+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.022+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.022+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.022+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.023+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.023+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.969+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.975+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.975+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.979+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.979+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.993+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.993+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.998+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.998+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.999+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.999+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:58:59.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.015+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.016+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.943+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.943+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.943+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.944+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.944+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.944+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.944+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.945+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.945+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.946+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.946+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.946+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.946+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.947+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.947+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.947+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.947+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.948+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.948+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.948+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.948+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.949+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.949+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.949+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.949+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.950+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.950+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.951+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.951+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.951+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.951+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.952+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.952+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.952+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.952+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.953+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.953+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.953+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.953+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.954+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.954+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.954+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.954+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.955+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.955+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.955+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.955+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.956+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.956+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.956+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.956+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.957+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.957+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.957+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.957+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.958+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.958+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.958+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.958+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.959+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.959+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.960+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.960+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.961+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.961+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.961+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.961+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.962+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.962+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.962+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.962+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.963+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.963+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.963+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.963+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.964+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.964+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.964+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.964+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.965+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.965+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.965+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.965+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.966+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.966+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.967+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.967+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.967+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.967+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.968+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.968+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.968+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.968+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.969+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.969+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.969+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.969+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.970+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.970+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.970+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.970+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.971+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.971+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.971+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.971+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.972+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.972+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.972+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.972+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.973+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.973+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.973+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.973+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.974+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.974+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.974+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.975+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.975+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.975+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.977+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.977+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.977+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.977+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.978+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.978+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.978+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.978+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.979+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.979+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.979+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.979+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.980+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.980+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.980+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.980+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.981+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.981+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.981+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.981+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.982+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.982+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.982+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.982+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.983+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.983+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.984+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.984+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.984+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.984+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.985+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.985+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.985+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.985+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.986+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.986+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.986+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.986+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.987+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.987+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.987+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.987+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.988+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.988+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.988+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.988+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.989+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.989+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.989+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.989+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.990+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.990+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.990+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.990+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.991+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.991+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.992+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.992+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.992+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.992+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.993+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.993+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.993+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.993+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.023+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.023+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.024+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.024+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.024+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.024+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.025+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.025+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.025+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.025+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.026+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.026+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.026+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.026+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.027+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.027+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.027+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.027+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.028+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.028+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.028+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.028+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.029+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.029+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.029+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.029+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.030+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.030+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.030+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.030+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.031+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.031+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.031+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.031+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.032+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.032+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.032+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.032+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.033+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.033+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.033+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.033+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.034+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.034+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.034+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.034+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.035+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.035+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.036+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.036+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.036+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.036+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.037+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.037+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.037+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.037+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.038+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.038+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.038+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.038+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.039+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.039+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.039+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.039+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.040+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.040+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.040+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.040+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.041+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.041+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.041+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.041+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.042+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.042+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.042+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.042+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.043+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.043+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.043+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.043+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.044+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.044+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.044+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.044+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.045+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.045+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.045+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.045+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.046+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.046+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.047+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.047+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.047+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.047+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.048+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.048+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.048+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.048+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.049+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.049+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.049+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.049+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.051+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.051+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.051+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.051+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.052+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.052+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.052+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.052+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.053+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.053+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.053+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.053+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.054+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.054+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.054+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.054+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.055+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.055+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.056+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.056+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.056+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.056+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.057+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.057+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.057+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.057+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.058+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.058+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.058+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.058+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.060+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.061+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.061+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.061+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.061+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.062+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.062+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.062+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.062+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.063+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.063+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.063+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.063+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.064+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.064+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.064+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.064+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.065+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.065+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.065+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.065+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.066+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.066+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.066+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.066+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.067+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.067+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.067+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.067+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.068+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.068+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.068+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.068+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.069+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.069+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.069+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.069+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.070+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.070+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.070+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.070+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.071+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.071+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.071+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.071+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.025+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.025+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.027+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.027+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.028+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.028+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.030+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.030+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.030+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.030+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.031+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.031+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.031+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.031+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.032+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.032+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.032+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.032+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.037+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.037+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.038+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.038+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.039+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.039+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.040+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.040+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.041+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.041+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.041+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.041+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.043+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.043+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.043+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.043+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.044+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.044+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.047+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.047+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.047+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.047+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.048+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.048+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.048+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.048+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.050+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.050+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.051+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.051+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.051+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.051+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.057+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.057+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.058+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.058+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.058+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.058+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.063+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.063+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.063+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.063+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.070+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.070+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.994+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.994+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.994+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.994+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.995+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.995+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.995+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.995+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.996+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.996+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.996+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.997+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.997+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.997+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.997+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.998+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.998+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.998+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.998+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.999+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:58:59.999+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.000+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.000+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.000+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.000+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.001+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.001+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.001+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.001+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.002+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.002+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.002+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.002+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.003+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.003+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.003+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.003+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.004+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.004+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.004+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.004+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.005+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.005+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.005+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.005+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.006+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.006+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.006+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.006+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.007+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.007+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.007+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.007+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.008+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.008+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.008+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.008+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.009+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.009+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.010+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.010+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.010+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.010+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.012+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.012+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.012+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.012+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.013+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.013+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.013+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.013+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.014+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.014+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.014+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.014+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.015+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.015+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.015+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.015+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.016+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.016+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.016+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.016+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.017+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.017+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.017+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.017+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.018+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.018+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.018+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.018+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.019+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.019+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.019+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.019+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.021+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.021+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.021+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.021+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.022+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.022+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.022+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.022+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.023+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.023+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.023+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.023+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.024+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.024+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.024+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.024+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.025+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.025+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.025+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.025+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.026+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.026+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.026+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.026+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.028+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.028+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.028+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.028+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.029+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.029+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.029+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.029+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.030+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.030+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.030+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.030+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.031+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.031+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.031+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.031+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.032+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.032+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.035+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.035+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.035+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.035+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.036+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.036+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.036+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.036+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.037+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.037+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.037+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.037+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.038+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.038+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.038+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.038+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.039+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.039+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.039+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.039+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.040+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.040+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.040+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.040+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.043+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.043+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.043+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.043+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.044+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.044+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.072+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.072+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.072+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.072+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.073+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.073+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.073+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.073+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.074+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.074+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.076+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.076+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.076+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.076+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.077+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.077+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.077+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.077+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.078+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.078+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.078+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.078+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.079+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.079+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.079+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.079+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.080+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.080+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.080+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.080+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.081+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.081+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.081+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.081+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.082+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.082+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.082+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.082+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.083+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.083+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.083+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.083+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.084+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.084+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.084+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.084+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.085+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.085+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.085+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.085+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.086+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.086+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.086+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.086+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.086+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.086+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.087+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.087+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.087+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.087+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.088+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.088+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.088+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.088+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.089+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.089+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.089+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.089+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.090+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.090+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.090+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.090+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.091+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.091+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.091+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.091+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.092+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.092+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.092+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.092+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.093+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.093+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.093+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.093+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.094+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.094+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.094+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.094+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.095+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.095+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.095+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.095+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.096+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.096+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.097+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.097+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.097+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.097+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.098+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.098+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.098+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.098+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.099+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.099+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.099+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.099+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.103+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.103+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.103+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.103+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.104+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.104+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.104+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.104+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.105+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.105+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.105+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.105+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.106+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.106+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.106+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.106+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.107+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.107+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.107+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.107+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.108+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.108+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.108+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.108+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.109+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.109+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.109+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.109+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.110+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.110+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.110+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.110+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.111+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.111+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.111+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.111+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.112+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.112+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.112+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.112+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.113+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.113+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.113+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.113+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.114+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.114+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.114+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.114+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.115+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.115+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.115+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.115+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.116+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.116+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.116+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.116+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.117+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.117+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.117+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.117+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.118+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.119+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.070+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.070+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.075+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.075+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.075+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.075+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.076+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.076+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.076+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.076+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.077+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.077+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.077+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.077+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.078+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.078+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.078+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.078+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.083+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.083+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.087+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.087+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.087+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.087+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.088+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.088+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.088+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.088+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.090+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.090+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.091+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.091+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.091+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.091+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.092+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.092+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.092+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.092+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.093+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.093+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.110+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.110+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.110+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.114+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.114+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.114+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.114+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.115+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.115+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.115+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.115+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.116+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.116+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.117+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.117+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.044+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.044+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.044+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.044+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.045+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.045+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.045+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.045+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.046+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.046+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.046+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.046+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.047+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.047+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.047+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.047+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.048+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.048+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.049+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.049+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.049+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.049+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.050+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.050+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.050+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.050+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.051+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.051+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.051+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.051+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.052+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.052+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.052+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.052+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.053+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.053+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.053+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.053+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.055+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.055+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.055+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.055+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.056+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.056+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.056+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.056+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.057+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.057+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.057+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.057+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.058+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.058+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.058+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.058+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.059+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.059+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.059+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.059+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.060+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.060+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.060+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.060+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.061+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.061+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.061+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.061+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.062+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.062+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.062+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.062+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.063+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.063+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.063+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.063+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.064+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.064+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.064+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.064+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.065+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.065+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.065+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.065+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.066+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.066+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.066+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.066+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.067+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.067+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.067+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.067+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.068+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.068+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.068+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.068+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.069+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.069+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.069+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.069+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.070+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.070+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.070+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.070+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.071+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.071+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.071+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.071+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.072+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.072+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.072+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.072+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.073+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.073+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.074+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.074+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.074+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.074+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.075+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.075+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.075+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.075+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.076+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.076+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.076+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.076+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.077+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.077+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.077+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.077+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.078+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.078+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.078+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.078+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.079+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.079+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.079+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.079+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.080+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.080+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.080+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.080+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.081+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.081+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.081+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.081+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.082+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.082+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.082+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.082+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.083+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.083+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.083+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.083+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.084+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.084+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.084+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.084+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.085+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.085+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.085+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.085+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.086+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.086+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.086+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.086+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.087+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.087+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.087+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.087+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.088+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.088+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.088+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.088+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.089+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.089+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.090+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.090+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.090+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.090+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.091+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.091+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.120+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.120+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.120+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.120+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.120+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.120+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.121+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.121+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.121+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.121+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.122+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.122+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.122+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.122+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.123+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.123+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.123+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.123+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.124+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.124+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.124+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.124+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.125+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.125+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.125+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.125+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.126+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.126+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.126+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.126+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.127+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.127+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.127+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.127+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.128+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.128+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.128+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.128+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.129+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.129+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.129+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.129+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.130+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.130+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.130+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.130+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.131+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.131+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.131+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.131+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.132+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.132+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.132+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.132+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.133+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.133+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.133+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.133+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.134+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.134+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.134+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.134+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.135+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.135+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.135+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.135+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.136+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.136+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.136+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.136+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.137+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.137+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.137+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.137+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.139+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.139+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.139+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.139+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.140+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.140+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.140+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.140+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.141+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.141+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.141+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.141+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.142+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.142+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.142+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.142+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.143+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.143+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.143+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.143+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.144+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.144+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.144+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.144+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.145+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.145+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.145+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.145+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.146+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.146+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.146+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.146+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.147+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.147+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.147+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.147+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.148+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.148+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.148+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.148+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.149+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.149+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.149+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.149+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.150+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.150+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.150+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.150+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.151+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.151+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.151+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.151+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.152+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.152+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.152+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.152+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.153+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.153+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.153+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.153+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.154+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.154+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.154+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.154+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.155+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.155+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.155+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.155+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.156+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.156+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.156+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.156+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.157+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.157+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.157+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.157+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.158+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.158+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.158+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.158+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.159+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.159+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.159+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.159+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.160+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.160+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.160+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.160+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.163+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.163+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.163+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.163+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.164+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.164+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.164+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.164+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.165+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.165+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.165+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.165+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.117+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.117+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.134+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.134+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.134+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.134+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.135+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.135+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.135+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.136+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.136+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.136+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.137+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.137+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.141+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.141+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.141+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.141+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.142+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.142+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.142+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.142+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.147+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.147+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.147+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.147+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.152+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.152+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.155+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.155+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.156+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.156+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.156+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.156+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.163+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.163+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.091+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.091+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.092+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.092+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.092+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.092+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.093+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.093+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.093+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.093+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.094+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.094+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.094+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.094+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.095+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.095+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.095+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.095+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.096+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.096+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.096+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.096+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.097+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.097+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.097+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.097+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.098+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.098+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.098+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.098+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.099+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.099+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.099+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.099+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.100+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.100+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.100+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.100+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.101+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.101+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.101+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.101+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.102+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.102+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.102+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.102+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.103+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.103+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.103+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.103+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.104+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.104+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.104+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.104+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.105+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.105+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.105+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.105+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.106+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.106+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.106+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.106+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.107+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.107+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.107+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.107+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.108+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.108+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.108+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.108+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.109+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.109+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.110+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.110+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.110+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.110+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.111+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.111+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.111+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.111+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.112+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.112+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.112+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.112+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.113+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.113+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.113+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.113+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.114+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.114+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.114+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.114+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.115+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.115+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.115+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.115+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.115+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.115+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.116+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.116+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.116+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.116+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.117+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.117+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.118+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.118+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.118+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.118+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.119+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.119+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.119+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.119+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.120+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.120+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.120+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.120+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.121+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.121+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.121+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.121+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.122+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.122+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.122+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.122+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.123+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.123+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.123+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.123+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.123+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.123+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.124+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.124+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.124+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.124+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.125+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.125+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.125+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.125+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.126+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.126+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.126+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.126+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.127+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.127+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.127+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.127+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.128+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.128+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.128+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.128+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.129+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.129+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.129+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.129+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.130+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.130+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.130+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.130+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.131+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.131+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.131+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.131+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.132+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.132+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.132+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.132+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.133+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.133+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.133+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.133+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.134+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.134+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.134+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.134+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.135+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.135+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.135+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.135+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.136+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.136+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.136+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.136+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.137+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.137+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.137+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.137+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.168+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.168+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.168+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.168+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.169+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.169+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.169+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.169+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.170+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.170+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.170+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.170+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.171+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.171+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.171+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.171+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.172+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.172+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.172+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.172+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.173+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.173+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.173+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.173+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.174+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.174+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.174+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.174+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.175+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.175+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.175+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.175+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.176+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.176+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.176+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.176+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.177+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.177+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.177+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.177+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.178+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.178+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.178+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.178+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.180+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.180+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.180+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.180+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.180+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.181+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.181+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.181+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.181+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.182+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.182+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.182+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.182+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.183+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.183+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.183+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.183+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.184+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.184+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.184+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.184+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.186+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.186+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.186+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.186+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.187+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.187+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.187+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.187+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.189+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.189+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.189+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.189+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.191+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.191+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.191+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.191+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.192+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.192+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.192+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.192+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.193+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.193+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.193+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.193+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.194+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.194+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.194+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.194+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.195+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.195+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.195+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.195+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.196+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.196+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.196+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.196+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.197+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.197+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.197+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.197+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.198+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.198+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.198+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.198+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.199+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.199+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.199+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.199+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.199+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.199+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.200+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.200+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.200+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.200+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.202+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.202+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.202+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.202+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.203+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.203+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.203+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.203+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.204+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.204+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.204+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.204+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.205+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.205+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.205+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.205+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.206+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.206+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.206+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.206+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.207+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.207+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.207+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.207+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.208+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.208+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.208+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.208+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.209+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.209+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.209+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.209+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.210+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.210+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.210+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.210+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.211+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.211+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.211+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.211+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.212+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.212+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.212+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.172+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.172+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.172+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.172+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.174+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.174+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.179+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.179+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.179+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.179+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.192+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.192+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.192+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.192+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.193+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.193+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.193+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.193+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.198+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.198+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.198+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.198+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.205+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.205+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.208+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.208+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.208+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.208+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.209+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.209+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.210+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.210+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.211+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.211+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.139+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.139+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.139+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.139+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.140+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.140+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.140+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.140+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.141+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.141+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.141+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.141+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.142+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.142+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.142+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.142+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.143+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.143+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.143+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.143+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.144+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.144+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.144+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.144+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.145+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.145+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.145+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.145+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.146+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.146+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.146+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.146+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.147+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.147+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.147+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.147+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.148+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.148+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.148+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.148+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.149+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.149+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.149+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.149+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.150+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.150+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.150+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.151+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.151+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.151+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.151+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.151+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.152+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.152+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.152+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.152+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.153+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.153+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.153+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.153+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.154+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.154+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.154+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.154+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.155+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.155+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.155+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.155+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.158+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.158+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.158+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.158+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.159+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.159+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.159+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.159+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.160+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.160+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.160+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.160+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.161+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.161+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.163+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.163+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.164+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.164+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.164+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.164+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.165+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.165+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.165+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.165+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.166+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.166+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.166+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.166+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.167+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.167+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.167+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.167+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.168+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.168+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.168+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.168+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.169+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.169+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.169+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.169+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.170+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.170+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.170+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.170+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.171+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.171+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.171+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.171+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.172+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.172+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.172+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.172+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.173+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.173+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.173+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.173+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.174+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.174+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.174+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.174+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.176+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.176+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.176+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.176+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.177+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.177+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.177+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.177+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.178+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.178+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.178+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.178+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.179+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.179+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.179+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.179+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.180+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.180+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.180+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.180+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.181+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.181+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.182+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.182+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.182+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.182+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.184+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.184+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.184+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.184+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.185+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.185+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.185+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.185+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.186+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.186+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.186+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.186+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.187+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.187+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.187+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.212+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.213+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.213+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.213+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.213+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.214+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.214+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.214+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.214+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.215+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.215+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.215+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.215+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.216+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.216+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.216+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.216+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.217+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.217+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.217+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.217+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.218+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.218+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.218+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.218+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.219+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.219+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.219+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.219+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.220+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.220+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.222+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.222+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.222+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.222+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.223+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.223+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.223+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.223+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.224+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.224+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.224+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.224+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.225+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.225+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.226+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.226+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.226+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.226+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.227+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.227+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.227+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.227+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.228+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.228+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.228+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.228+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.229+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.229+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.229+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.229+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.230+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.230+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.230+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.230+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.230+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.231+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.231+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.231+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.231+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.232+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.232+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.232+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.233+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.233+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.233+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.233+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.234+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.234+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.234+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.234+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.235+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.235+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.235+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.235+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.236+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.236+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.236+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.236+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.236+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.236+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.237+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.237+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.237+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.237+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.238+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.238+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.238+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.238+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.239+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.239+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.239+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.239+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.240+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.240+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.240+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.240+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.241+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.241+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.241+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.241+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.242+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.242+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.242+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.242+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.243+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.243+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.243+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.243+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.244+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.244+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.244+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.244+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.245+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.245+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.245+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.245+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.246+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.246+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.246+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.246+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.246+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.246+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.247+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.247+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.247+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.247+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.248+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.248+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.248+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.248+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.249+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.249+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.249+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.249+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.250+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.250+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.250+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.250+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.251+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.251+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.251+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.251+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.252+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.252+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.252+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.252+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.253+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.253+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.253+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.253+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.254+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.254+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.254+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.254+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.255+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.255+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.255+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.255+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.256+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.256+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.256+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.256+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.257+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.257+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.257+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.257+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.258+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.258+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.258+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.221+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.221+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.221+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.238+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.238+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.238+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.238+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.241+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.241+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.241+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.241+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.242+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.242+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.243+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.243+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.244+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.244+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.244+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.244+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.246+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.246+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.187+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.189+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.189+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.189+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.189+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.190+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.190+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.190+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.190+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.191+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.191+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.191+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.191+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.193+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.193+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.193+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.193+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.195+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.195+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.197+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.197+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.197+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.197+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.198+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.198+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.198+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.198+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.199+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.199+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.199+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.199+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.200+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.200+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.200+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.200+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.201+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.201+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.201+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.201+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.202+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.202+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.202+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.202+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.203+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.203+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.203+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.203+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.204+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.204+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.204+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.204+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.205+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.205+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.205+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.205+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.206+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.206+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.206+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.206+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.207+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.207+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.207+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.207+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.208+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.208+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.208+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.208+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.208+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.208+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.209+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.209+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.209+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.209+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.210+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.210+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.210+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.210+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.211+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.211+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.211+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.211+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.213+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.213+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.214+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.214+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.216+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.216+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.216+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.216+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.217+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.217+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.217+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.217+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.218+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.218+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.218+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.218+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.218+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.218+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.219+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.219+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.219+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.219+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.220+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.220+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.220+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.220+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.221+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.221+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.221+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.221+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.222+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.222+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.222+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.222+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.223+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.223+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.223+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.223+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.224+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.224+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.224+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.224+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.225+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.225+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.225+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.225+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.227+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.227+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.227+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.227+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.228+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.228+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.228+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.228+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.229+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.229+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.229+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.229+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.230+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.230+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.230+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.230+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.231+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.231+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.231+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.231+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.231+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.232+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.232+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.232+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.233+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.233+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.233+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.233+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.234+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.234+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.234+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.234+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.235+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.258+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.259+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.259+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.259+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.259+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.260+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.260+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.260+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.260+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.261+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.261+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.261+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.261+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.262+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.262+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.262+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.262+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.263+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.263+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.263+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.263+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.265+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.265+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.265+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.265+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.265+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.265+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.266+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.266+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.266+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.266+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.267+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.267+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.267+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.267+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.269+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.269+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.269+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.269+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.270+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.270+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.270+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.270+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.271+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.271+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.271+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.271+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.272+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.272+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.272+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.272+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.273+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.273+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.273+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.273+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.274+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.274+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.274+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.274+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.276+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.276+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.276+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.276+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.276+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.276+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.277+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.277+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.278+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.278+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.278+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.278+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.279+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.279+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.279+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.279+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.280+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.280+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.280+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.280+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.281+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.281+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.281+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.281+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.282+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.282+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.282+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.282+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.283+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.283+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.283+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.283+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.284+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.284+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.284+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.284+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.284+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.284+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.285+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.285+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.285+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.285+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.286+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.286+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.286+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.286+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.287+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.287+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.287+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.287+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.288+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.288+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.288+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.288+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.289+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.289+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.289+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.289+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.290+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.290+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.292+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.292+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.292+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.292+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.293+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.293+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.293+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.293+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.294+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.294+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.294+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.294+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.295+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.295+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.295+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.295+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.296+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.296+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.296+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.296+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.297+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.297+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.297+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.297+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.298+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.298+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.298+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.298+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.299+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.299+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.299+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.299+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.300+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.300+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.300+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.300+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.301+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.301+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.301+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.301+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.302+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.302+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.302+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.302+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.303+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.303+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.303+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.303+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.304+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.304+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.304+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.304+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.305+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.305+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.305+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.305+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.306+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.306+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.269+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.269+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.269+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.269+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.275+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.275+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.292+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.292+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.293+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.293+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.293+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.294+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.294+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.294+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.294+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.295+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.295+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.295+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.295+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.296+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.296+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.296+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.296+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.301+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.301+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.308+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.308+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.308+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.308+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.235+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.235+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.235+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.235+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.236+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.236+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.236+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.236+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.236+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.237+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.237+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.237+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.237+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.238+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.238+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.238+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.238+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.239+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.239+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.239+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.239+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.240+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.240+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.240+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.240+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.241+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.241+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.241+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.241+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.242+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.242+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.242+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.242+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.242+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.242+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.243+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.243+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.243+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.243+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.244+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.244+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.244+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.244+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.245+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.245+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.245+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.245+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.246+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.246+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.246+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.246+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.247+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.247+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.247+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.247+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.248+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.248+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.248+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.248+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.249+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.249+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.249+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.249+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.250+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.250+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.250+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.250+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.251+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.251+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.251+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.251+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.252+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.252+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.252+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.252+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.253+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.253+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.253+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.253+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.254+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.254+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.254+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.254+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.255+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.255+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.255+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.255+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.256+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.256+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.256+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.256+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.257+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.257+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.257+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.257+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.258+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.258+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.258+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.258+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.259+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.259+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.259+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.259+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.260+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.260+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.260+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.260+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.260+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.260+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.261+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.261+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.261+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.261+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.262+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.262+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.262+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.262+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.263+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.263+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.263+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.263+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.264+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.264+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.264+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.264+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.265+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.265+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.265+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.265+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.266+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.266+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.266+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.266+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.267+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.267+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.267+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.267+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.268+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.268+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.268+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.268+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.269+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.269+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.269+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.269+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.270+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.270+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.270+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.270+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.271+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.271+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.271+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.271+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.272+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.272+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.272+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.272+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.273+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.273+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.273+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.273+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.274+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.274+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.274+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.274+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.274+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.274+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.275+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.275+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.275+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.275+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.276+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.276+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.276+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.276+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.277+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.277+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.277+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.277+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.278+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.278+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.278+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.278+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.279+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.279+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.279+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.279+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.280+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.280+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.306+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.306+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.307+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.307+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.307+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.307+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.308+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.308+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.308+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.308+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.309+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.309+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.309+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.309+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.310+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.310+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.310+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.310+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.311+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.311+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.311+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.311+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.312+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.312+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.312+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.312+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.313+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.313+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.313+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.313+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.314+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.314+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.314+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.314+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.315+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.315+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.315+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.315+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.316+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.316+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.316+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.316+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.317+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.317+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.318+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.318+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.318+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.318+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.319+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.319+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.319+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.319+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.320+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.320+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.320+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.320+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.321+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.321+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.321+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.321+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.322+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.322+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.322+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.322+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.323+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.323+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.323+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.323+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.323+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.323+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.324+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.324+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.324+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.324+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.325+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.325+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.325+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.325+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.326+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.326+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.326+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.326+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.327+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.327+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.327+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.327+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.328+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.328+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.328+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.328+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.329+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.329+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.329+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.329+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.330+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.330+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.330+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.330+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.331+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.331+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.331+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.331+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.332+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.332+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.332+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.332+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.333+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.333+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.333+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.333+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.334+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.334+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.334+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.334+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.335+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.335+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.335+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.335+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.336+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.336+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.336+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.336+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.337+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.337+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.337+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.337+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.338+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.338+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.338+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.338+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.338+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.339+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.339+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.339+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.339+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.339+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.340+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.340+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.340+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.340+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.341+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.341+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.341+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.341+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.342+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.342+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.342+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.342+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.343+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.343+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.343+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.343+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.344+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.344+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.344+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.344+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.345+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.345+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.345+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.345+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.346+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.346+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.346+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.346+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.347+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.347+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.347+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.347+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.348+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.348+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.348+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.348+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.348+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.348+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.349+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.349+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.349+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.349+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.350+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.350+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.350+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.350+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.351+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.351+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.351+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.351+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.352+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.352+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.312+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.312+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.314+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.314+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.314+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.314+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.315+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.315+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.337+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.337+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.280+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.280+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.282+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.282+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.282+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.282+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.283+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.283+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.283+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.283+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.284+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.284+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.284+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.284+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.285+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.285+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.285+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.285+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.286+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.286+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.286+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.286+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.287+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.287+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.287+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.287+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.288+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.288+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.288+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.288+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.289+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.289+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.289+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.289+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.290+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.290+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.290+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.290+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.292+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.292+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.293+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.293+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.293+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.293+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.294+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.294+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.294+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.294+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.295+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.295+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.295+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.295+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.296+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.296+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.296+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.296+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.297+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.297+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.297+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.297+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.298+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.298+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.298+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.298+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.299+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.299+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.299+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.299+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.300+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.300+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.300+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.300+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.300+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.300+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.301+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.301+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.301+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.302+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.302+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.302+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.303+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.303+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.303+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.303+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.304+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.304+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.304+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.304+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.305+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.305+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.305+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.305+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.306+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.306+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.306+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.306+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.307+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.307+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.307+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.307+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.308+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.308+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.308+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.308+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.309+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.309+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.309+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.309+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.309+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.309+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.310+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.310+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.310+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.311+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.311+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.311+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.311+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.311+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.312+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.312+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.312+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.312+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.313+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.313+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.313+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.313+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.314+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.314+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.314+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.314+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.315+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.315+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.315+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.315+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.316+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.316+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.316+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.316+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.317+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.317+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.317+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.317+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.318+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.318+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.318+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.318+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.319+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.319+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.319+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.319+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.320+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.320+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.320+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.320+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.321+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.321+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.321+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.321+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.322+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.322+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.322+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.322+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.323+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.323+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.323+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.323+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.324+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.324+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.324+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.324+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.325+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.325+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.325+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.325+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.326+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.326+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.326+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.326+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.327+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.327+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.327+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.327+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.328+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.328+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.352+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.352+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.353+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.353+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.353+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.353+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.354+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.354+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.354+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.354+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.355+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.355+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.355+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.355+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.356+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.356+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.356+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.356+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.357+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.357+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.357+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.357+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.358+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.358+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.358+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.358+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.359+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.359+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.359+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.359+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.360+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.360+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.360+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.360+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.361+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.361+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.361+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.361+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.362+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.362+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.363+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.363+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.363+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.363+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.364+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.364+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.364+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.364+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.365+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.365+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.365+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.365+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.366+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.366+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.366+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.366+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.367+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.367+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.367+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.367+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.368+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.368+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.368+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.368+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.369+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.369+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.370+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.370+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.372+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.372+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.372+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.372+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.373+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.373+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.373+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.373+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.374+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.374+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.374+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.374+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.375+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.375+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.375+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.375+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.376+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.376+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.376+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.376+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.377+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.377+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.377+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.377+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.378+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.378+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.378+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.378+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.379+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.379+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.379+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.379+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.380+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.380+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.380+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.380+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.381+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.381+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.381+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.381+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.382+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.382+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.382+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.382+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.383+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.383+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.385+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.385+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.385+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.385+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.386+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.386+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.386+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.386+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.387+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.387+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.387+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.387+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.387+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.387+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.388+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.388+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.388+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.388+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.389+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.389+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.389+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.389+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.390+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.390+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.390+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.390+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.391+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.391+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.391+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.391+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.392+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.392+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.392+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.392+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.393+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.393+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.393+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.393+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.394+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.394+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.394+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.394+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.395+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.395+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.397+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.397+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.397+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.397+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.398+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.398+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.398+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.398+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.399+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.399+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.399+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.399+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.400+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.400+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.400+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.400+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.401+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.401+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.401+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.401+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.401+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.401+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.402+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.402+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.402+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.402+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.403+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.403+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.403+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.403+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.386+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.386+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.394+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.394+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.328+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.328+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.329+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.329+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.329+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.329+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.329+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.329+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.330+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.330+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.330+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.330+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.331+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.331+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.331+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.331+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.332+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.332+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.332+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.333+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.333+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.333+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.333+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.333+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.334+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.335+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.335+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.335+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.335+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.336+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.336+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.336+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.336+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.337+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.337+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.337+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.337+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.338+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.338+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.338+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.338+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.339+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.339+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.339+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.339+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.340+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.340+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.340+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.340+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.341+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.341+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.341+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.341+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.342+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.342+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.342+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.342+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.343+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.343+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.343+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.343+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.344+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.344+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.344+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.344+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.345+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.345+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.345+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.345+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.346+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.346+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.346+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.346+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.347+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.347+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.347+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.347+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.348+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.348+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.348+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.348+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.349+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.349+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.349+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.349+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.349+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.349+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.350+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.350+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.350+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.350+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.351+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.351+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.351+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.351+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.352+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.352+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.352+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.352+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.353+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.353+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.353+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.353+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.354+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.354+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.354+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.354+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.355+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.355+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.355+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.355+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.356+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.356+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.356+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.356+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.357+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.357+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.357+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.357+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.358+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.358+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.359+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.359+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.359+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.359+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.361+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.361+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.362+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.362+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.362+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.362+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.363+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.363+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.363+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.363+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.364+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.364+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.364+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.364+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.365+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.365+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.365+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.365+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.366+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.366+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.366+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.366+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.367+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.367+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.367+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.367+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.368+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.368+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.368+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.368+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.369+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.369+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.369+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.369+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.370+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.370+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.370+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.370+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.371+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.371+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.371+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.371+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.372+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.372+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.372+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.373+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.373+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.373+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.373+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.373+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.374+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.374+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.374+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.374+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.404+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.404+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.405+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.405+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.405+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.405+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.405+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.405+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.406+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.406+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.406+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.406+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.407+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.407+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.408+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.408+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.408+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.408+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.408+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.408+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.409+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.409+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.409+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.409+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.410+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.410+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.410+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.410+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.411+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.411+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.411+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.411+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.412+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.412+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.412+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.412+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.413+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.413+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:00.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.413+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:00.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.413+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.414+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.414+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.414+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.414+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.415+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.415+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.415+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.415+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.416+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.416+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.416+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.416+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.417+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.417+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.417+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.417+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.418+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.418+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.418+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.418+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.419+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.419+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.419+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.419+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.421+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.421+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.421+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.421+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.421+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.421+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.421+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.421+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.422+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.422+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.422+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.422+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.423+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.423+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.423+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.423+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.426+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.426+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.427+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.427+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.427+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.427+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.427+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.427+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.427+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.427+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.428+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.428+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.428+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.428+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.429+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.429+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.429+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.429+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.430+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.430+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.430+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.430+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.431+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.431+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.431+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.431+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.432+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.432+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.432+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.432+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.433+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.433+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.434+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.434+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.434+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.434+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.435+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.435+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.435+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.435+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.436+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.436+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.436+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.436+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.437+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.437+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.437+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.437+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.439+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.439+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.439+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.439+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.439+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.439+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.439+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.439+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.441+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.441+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.441+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.441+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.441+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.441+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.441+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.441+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.443+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.443+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.443+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.443+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.443+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.443+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.443+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.443+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.445+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.445+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.445+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.445+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.445+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.445+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.447+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.447+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.447+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.447+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.448+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.448+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.448+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.448+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.449+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.449+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.449+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.449+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.450+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.450+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.450+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.450+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.450+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.450+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.450+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.450+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.452+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.452+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.452+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.452+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.453+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.453+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.453+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.453+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.454+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.454+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.454+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.411+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.411+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.419+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.419+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.424+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.424+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.434+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.434+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.434+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.434+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.435+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.435+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.435+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.435+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.437+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.437+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.437+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.437+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.439+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.439+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.445+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.445+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.445+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.445+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.447+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.447+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.448+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.448+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.448+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.448+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.454+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.454+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.375+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.375+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.375+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.375+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.376+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.376+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.376+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.376+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.377+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.377+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.377+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.377+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.378+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.378+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.378+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.378+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.379+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.379+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.379+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.379+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.380+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.380+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.380+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.380+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.381+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.381+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.381+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.381+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.382+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.382+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.382+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.382+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.383+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.383+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.383+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.383+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.384+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.384+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.384+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.384+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.385+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.385+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.385+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.385+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.386+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.386+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.386+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.386+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.387+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.387+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.387+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.387+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.388+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.388+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.388+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.388+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.389+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.389+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.389+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.389+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.391+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.391+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.391+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.391+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.392+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.392+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.392+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.392+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.393+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.393+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.393+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.393+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.394+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.394+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.394+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.394+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.395+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.395+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.395+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.395+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.396+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.396+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.396+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.396+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.397+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.397+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.397+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.397+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.398+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.398+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.398+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.398+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.399+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.399+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.399+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.399+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.400+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.400+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.400+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.400+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.401+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.401+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.401+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.401+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.403+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.405+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.405+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.405+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.405+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.406+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.406+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.406+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.406+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.407+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.407+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.407+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.407+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.408+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.408+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.408+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.408+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.409+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.409+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.409+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.409+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.410+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.410+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.410+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.410+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.411+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.411+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.411+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.411+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.413+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.413+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.413+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.413+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.414+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.414+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.414+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.414+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.415+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.415+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.415+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.415+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.416+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.416+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.416+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.416+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.418+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.418+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.418+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.418+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.419+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.419+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.419+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.419+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.422+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.422+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.422+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.422+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.423+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.423+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.423+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.454+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.455+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.455+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.455+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.455+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.456+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.456+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.456+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.456+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.457+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.457+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.457+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.457+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.458+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.458+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.458+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.458+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.459+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.459+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.459+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.459+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.460+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.460+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.460+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.460+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.461+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.461+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.461+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.461+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.462+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.462+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.462+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.462+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.463+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.463+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.463+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.463+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.464+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.464+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.464+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.464+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.465+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.465+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.465+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.465+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.466+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.466+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.466+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.466+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.467+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.467+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.467+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.467+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.467+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.467+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.468+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.468+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.468+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.468+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.469+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.469+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.470+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.470+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.470+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.470+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.471+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.471+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.471+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.471+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.472+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.472+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.472+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.472+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.473+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.473+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.473+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.473+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.474+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.474+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.474+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.474+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.474+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.474+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.475+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.475+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.475+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.475+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.476+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.476+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.476+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.476+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.477+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.477+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.477+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.477+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.478+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.478+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.478+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.478+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.479+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.479+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.479+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.479+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.480+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.480+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.480+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.480+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.481+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.481+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.481+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.481+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.482+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.482+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.482+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.482+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.483+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.483+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.483+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.483+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.485+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.485+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.486+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.486+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.486+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.486+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.487+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.487+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.487+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.487+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.488+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.488+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.488+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.488+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.489+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.489+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.489+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.489+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.490+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.490+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.490+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.490+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.491+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.491+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.491+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.491+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.493+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.493+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.494+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.494+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.494+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.494+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.495+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.495+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.495+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.495+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.496+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.496+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.496+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.496+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.497+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.497+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.497+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.497+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.498+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.498+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.498+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.498+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.499+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.499+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.499+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.499+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.500+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.500+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.500+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.500+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.501+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.501+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.501+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.501+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.502+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.502+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.502+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.502+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.503+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.503+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.503+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.503+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.504+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.462+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.462+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.491+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.491+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.491+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.491+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.492+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.492+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.501+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.501+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.501+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.501+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.502+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.502+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.502+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.502+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.503+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.503+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.503+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.503+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.423+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.424+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.424+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.424+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.424+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.426+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.426+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.426+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.426+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.428+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.428+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.428+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.428+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.429+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.429+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.429+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.429+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.430+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.430+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.430+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.430+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.432+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.432+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.433+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.433+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.433+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.433+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.434+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.434+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.434+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.434+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.436+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.436+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.436+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.436+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.437+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.437+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.437+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.437+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.439+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.439+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.439+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.439+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.439+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.439+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.439+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.439+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.443+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.443+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.445+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.445+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.445+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.445+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.445+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.445+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.445+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.445+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.447+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.447+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.447+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.447+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.448+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.448+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.448+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.448+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.449+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.449+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.449+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.449+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.450+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.450+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.450+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.450+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.450+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.450+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.450+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.450+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.452+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.452+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.452+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.452+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.453+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.453+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.453+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.453+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.454+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.454+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.454+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.454+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.455+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.455+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.456+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.456+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.456+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.456+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.457+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.457+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.457+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.457+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.458+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.458+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.458+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.458+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.459+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.459+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.459+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.459+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.460+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.460+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.460+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.460+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.461+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.461+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.461+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.461+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.462+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.462+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.462+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.462+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.463+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.463+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.463+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.463+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.464+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.464+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.464+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.464+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.465+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.465+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.465+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.465+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.466+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.466+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.466+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.466+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.467+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.467+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.467+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.467+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.468+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.468+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.468+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.468+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.470+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.470+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.470+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.470+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.471+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.471+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.471+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.471+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.472+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.504+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.504+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.504+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.505+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.505+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.506+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.506+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.506+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.506+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.507+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.507+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.507+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.507+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.507+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.508+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.508+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.508+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.508+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.508+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.509+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.509+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.509+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.509+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.510+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.510+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.510+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.511+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.511+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.511+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.512+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.512+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.512+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.512+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.513+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.513+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.513+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.513+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.514+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.514+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.514+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.514+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.515+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.515+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.517+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.517+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.517+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.517+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.518+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.518+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.518+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.518+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.519+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.519+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.519+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.519+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.520+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.520+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.521+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.521+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.521+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.521+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.522+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.522+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.522+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.522+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.523+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.523+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.523+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.523+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.524+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.524+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.524+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.524+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.525+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.525+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.525+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.525+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.526+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.526+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.526+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.526+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.527+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.527+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.527+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.527+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.528+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.528+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.528+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.528+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.528+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.528+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.529+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.529+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.529+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.530+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.530+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.530+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.531+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.531+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.531+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.531+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.532+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.532+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.532+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.532+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.533+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.533+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.533+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.533+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.534+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.534+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.534+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.534+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.535+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.535+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.535+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.535+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.536+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.536+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.536+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.536+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.537+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.537+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.537+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.537+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.538+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.538+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.538+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.538+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.539+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.539+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.539+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.539+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.539+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.539+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.540+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.540+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.541+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.541+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.541+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.541+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.542+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.542+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.542+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.542+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.543+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.543+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.543+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.543+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.544+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.544+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.544+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.544+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.545+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.545+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.545+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.545+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.546+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.546+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.546+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.546+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.547+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.547+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.547+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.547+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.548+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.548+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.548+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.548+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.549+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.549+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.549+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.549+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.549+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.550+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.550+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.550+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.550+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.550+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.551+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.551+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.551+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.551+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.552+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.552+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.552+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.552+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.553+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.504+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.504+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.541+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.541+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.541+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.541+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.472+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.472+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.472+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.473+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.473+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.473+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.473+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.474+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.474+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.474+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.474+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.475+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.475+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.476+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.476+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.476+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.476+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.477+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.477+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.477+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.477+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.478+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.478+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.478+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.478+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.479+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.479+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.479+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.479+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.480+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.480+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.480+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.480+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.481+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.481+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.481+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.481+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.482+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.482+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.483+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.483+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.483+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.483+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.484+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.484+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.484+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.484+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.485+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.485+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.485+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.485+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.487+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.487+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.487+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.487+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.488+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.488+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.490+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.490+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.490+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.490+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.491+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.491+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.491+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.491+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.492+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.492+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.492+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.493+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.493+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.493+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.494+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.494+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.494+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.494+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.495+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.495+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.495+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.495+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.496+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.496+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.496+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.496+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.497+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.497+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.497+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.497+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.498+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.498+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.498+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.498+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.499+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.499+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.499+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.499+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.500+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.500+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.500+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.500+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.501+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.501+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.501+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.501+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.502+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.502+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.502+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.502+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.503+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.503+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.503+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.503+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.504+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.504+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.504+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.504+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.505+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.505+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.505+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.505+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.506+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.506+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.506+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.506+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.507+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.507+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.507+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.507+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.508+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.508+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.508+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.508+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.509+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.509+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.509+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.509+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.510+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.510+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.511+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.511+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.511+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.511+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.512+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.512+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.512+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.512+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.513+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.513+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.513+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.513+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.514+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.514+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.514+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.514+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.515+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.515+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.515+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.515+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.516+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.516+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.517+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.517+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.517+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.517+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.518+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.518+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.518+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.518+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.519+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.519+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.519+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.519+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.520+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.520+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.520+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.520+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.521+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.521+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.521+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.521+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.522+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.522+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.522+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.522+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.523+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.553+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.553+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.553+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.554+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.554+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.554+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.554+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.555+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.555+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.555+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.555+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.556+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.556+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.556+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.556+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.557+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.557+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.557+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.557+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.558+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.558+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.558+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.558+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.559+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.559+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.559+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.559+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.560+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.560+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.560+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.560+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.561+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.561+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.561+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.561+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.562+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.562+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.562+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.562+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.563+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.563+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.563+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.563+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.564+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.564+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.564+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.564+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.565+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.565+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.565+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.565+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.566+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.566+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.566+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.566+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.567+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.567+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.567+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.567+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.568+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.568+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.568+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.568+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.569+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.569+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.569+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.569+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.570+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.570+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.570+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.570+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.571+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.571+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.571+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.571+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.572+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.572+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.572+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.572+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.573+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.573+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.573+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.573+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.574+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.574+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.575+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.575+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.575+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.575+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.575+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.575+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.576+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.576+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.576+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.576+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.577+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.577+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.578+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.578+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.578+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.578+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.578+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.578+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.579+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.579+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.579+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.579+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.580+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.580+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.580+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.580+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.581+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.581+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.581+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.581+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.582+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.582+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.582+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.582+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.583+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.583+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.583+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.583+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.584+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.584+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.584+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.584+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.585+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.585+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.585+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.585+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.586+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.586+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.586+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.586+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.587+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.587+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.589+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.589+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.590+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.590+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.590+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.590+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.591+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.591+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.591+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.591+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.592+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.592+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.592+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.592+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.593+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.593+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.593+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.593+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.594+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.594+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.594+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.594+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.595+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.595+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.595+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.595+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.596+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.596+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.596+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.596+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.597+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.597+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.597+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.597+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.598+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.598+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.598+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.598+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.599+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.599+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.599+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.599+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.600+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.600+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.600+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.600+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.601+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.601+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.601+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.601+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.602+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.602+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.569+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.569+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.572+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.523+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.523+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.523+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.524+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.524+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.524+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.524+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.525+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.525+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.525+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.525+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.526+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.526+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.526+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.526+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.527+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.527+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.527+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.527+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.528+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.528+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.528+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.528+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.529+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.529+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.529+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.529+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.530+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.530+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.530+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.530+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.531+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.531+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.531+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.531+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.532+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.532+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.532+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.532+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.533+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.533+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.533+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.533+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.534+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.534+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.534+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.534+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.535+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.535+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.535+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.535+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.536+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.536+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.536+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.536+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.537+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.537+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.537+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.537+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.538+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.538+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.538+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.538+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.539+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.539+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.539+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.539+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.540+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.540+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.541+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.541+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.541+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.541+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.542+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.542+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.542+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.542+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.543+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.543+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.543+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.543+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.544+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.544+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.544+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.544+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.545+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.545+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.545+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.545+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.546+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.546+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.546+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.546+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.547+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.547+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.547+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.547+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.548+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.548+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.548+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.548+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.549+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.549+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.549+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.549+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.550+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.550+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.550+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.550+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.550+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.551+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.551+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.551+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.551+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.551+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.552+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.552+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.552+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.552+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.553+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.553+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.553+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.553+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.554+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.554+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.554+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.554+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.555+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.555+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.555+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.555+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.556+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.556+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.556+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.556+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.557+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.557+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.557+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.557+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.558+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.558+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.558+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.558+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.559+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.559+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.559+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.559+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.560+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.560+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.560+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.560+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.561+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.561+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.561+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.561+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.562+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.562+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.562+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.562+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.563+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.563+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.563+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.563+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.564+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.564+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.564+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.564+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.565+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.565+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.565+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.565+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.566+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.566+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.566+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.566+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.567+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.567+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.567+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.567+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.568+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.568+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.568+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.568+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.569+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.569+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.569+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.569+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.570+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.570+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.602+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.602+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.603+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.603+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.603+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.603+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.604+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.604+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.604+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.604+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.605+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.605+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.605+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.605+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.606+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.606+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.606+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.606+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.607+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.607+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.607+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.607+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.608+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.608+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.608+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.608+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.609+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.609+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.609+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.609+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.610+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.610+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.610+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.610+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.611+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.611+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.611+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.611+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.612+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.612+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.612+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.612+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.613+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.613+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.613+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.613+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.614+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.614+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.614+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.614+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.615+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.615+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.615+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.615+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.616+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.616+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.616+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.616+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.617+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.617+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.617+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.617+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.618+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.618+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.618+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.618+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.619+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.619+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.619+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.619+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.620+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.620+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.620+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.620+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.621+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.621+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.621+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.621+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.622+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.622+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.622+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.622+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.623+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.623+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.623+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.623+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.624+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.624+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.625+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.625+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.625+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.625+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.626+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.626+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.626+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.626+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.627+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.627+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.627+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.627+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.628+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.628+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.628+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.628+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.629+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.629+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.629+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.629+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.630+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.630+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.630+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.630+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.631+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.631+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.631+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.631+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.632+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.632+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.633+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.633+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.633+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.633+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.634+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.634+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.634+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.634+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.637+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.637+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.637+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.637+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.638+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.638+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.638+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.638+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.639+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.639+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.639+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.639+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.640+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.640+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.640+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.640+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.641+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.641+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.641+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.641+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.642+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.642+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.642+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.642+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.643+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.643+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.643+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.643+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.644+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.644+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.644+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.644+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.646+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.646+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.647+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.647+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.647+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.647+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.648+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.648+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.648+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.648+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.649+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.649+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.649+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.649+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.650+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.650+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.650+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.650+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.651+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.651+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.652+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.652+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.652+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.652+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.653+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.653+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.653+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.653+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.615+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.615+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.615+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.615+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.618+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.618+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.619+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.619+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.627+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.627+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.628+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.628+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.630+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.630+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.630+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.630+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.570+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.570+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.571+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.571+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.571+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.571+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.572+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.572+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.572+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.572+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.573+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.573+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.573+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.573+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.574+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.574+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.574+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.574+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.575+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.575+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.575+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.575+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.576+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.576+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.576+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.576+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.577+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.577+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.577+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.577+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.578+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.578+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.578+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.578+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.579+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.579+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.579+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.579+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.580+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.580+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.580+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.580+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.581+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.581+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.581+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.581+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.582+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.582+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.582+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.582+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.583+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.583+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.583+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.583+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.584+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.584+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.584+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.584+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.585+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.585+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.585+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.585+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.586+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.586+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.586+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.586+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.587+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.587+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.587+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.587+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.588+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.588+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.589+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.589+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.589+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.589+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.590+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.590+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.590+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.590+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.591+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.591+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.591+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.591+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.592+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.592+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.592+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.592+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.593+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.593+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.593+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.593+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.594+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.594+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.594+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.594+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.595+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.595+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.595+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.595+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.596+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.596+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.596+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.596+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.597+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.597+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.597+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.597+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.598+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.598+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.598+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.598+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.599+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.599+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.599+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.599+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.600+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.600+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.600+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.600+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.601+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.601+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.601+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.601+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.602+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.602+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.602+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.602+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.603+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.603+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.603+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.603+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.604+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.604+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.604+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.604+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.605+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.605+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.605+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.605+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.606+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.606+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.606+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.606+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.607+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.607+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.607+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.607+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.608+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.608+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.608+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.608+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.609+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.609+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.609+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.609+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.610+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.610+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.610+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.610+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.611+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.611+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.611+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.611+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.612+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.612+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.612+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.612+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.613+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.613+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.613+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.613+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.614+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.614+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.614+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.614+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.615+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.615+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.615+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.615+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.616+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.616+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.616+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.616+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.617+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.617+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.617+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.617+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.654+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.654+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.654+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.654+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.654+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.654+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.655+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.655+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.655+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.655+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.656+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.656+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.657+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.657+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.657+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.657+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.658+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.658+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.658+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.658+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.659+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.659+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.659+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.659+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.660+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.660+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.660+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.660+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.660+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.660+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.661+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.661+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.662+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.662+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.662+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.662+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.663+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.663+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.663+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.663+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.664+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.664+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.664+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.664+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.665+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.665+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.665+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.665+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.666+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.666+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.666+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.666+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.667+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.667+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.667+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.667+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.668+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.668+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.668+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.668+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.669+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.669+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.669+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.669+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.670+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.670+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.670+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.670+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.671+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.671+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.671+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.671+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.672+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.672+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.672+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.672+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.673+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.673+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.673+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.673+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.674+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.674+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.674+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.674+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.676+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.676+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.676+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.676+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.677+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.677+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.677+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.677+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.679+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.679+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.679+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.679+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.680+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.680+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.681+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.681+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.681+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.681+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.682+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.682+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.682+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.682+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.683+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.683+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.683+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.683+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.685+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.685+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.685+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.685+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.686+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.686+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.686+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.686+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.687+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.687+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.687+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.687+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.688+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.688+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.689+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.689+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.689+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.689+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.690+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.690+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.690+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.690+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.691+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.691+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.692+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.692+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.692+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.692+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.693+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.693+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.693+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.693+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.694+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.694+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.694+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.694+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.695+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.695+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.695+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.695+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.696+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.696+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.698+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.698+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.698+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.698+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.699+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.699+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.699+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.699+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.700+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.700+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.700+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.700+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.648+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.648+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.648+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.648+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.666+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.666+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.666+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.666+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.666+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.684+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.684+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.684+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.684+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.685+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.685+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.685+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.685+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.688+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.688+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.688+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.688+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.689+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.689+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.689+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.689+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.618+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.618+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.618+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.618+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.619+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.619+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.619+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.619+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.620+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.620+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.620+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.620+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.621+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.621+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.621+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.621+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.622+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.622+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.622+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.622+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.623+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.623+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.623+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.623+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.624+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.624+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.624+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.624+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.625+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.625+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.625+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.625+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.626+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.626+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.626+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.626+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.627+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.627+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.627+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.627+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.628+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.628+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.630+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.630+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.630+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.630+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.631+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.631+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.631+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.631+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.632+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.632+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.633+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.633+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.633+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.633+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.633+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.634+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.634+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.634+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.634+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.634+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.635+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.635+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.636+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.636+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.636+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.636+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.637+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.637+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.637+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.637+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.638+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.638+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.640+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.640+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.640+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.640+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.641+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.641+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.641+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.641+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.642+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.642+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.642+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.642+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.643+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.643+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.643+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.643+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.644+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.644+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.644+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.644+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.645+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.645+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.645+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.645+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.646+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.646+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.646+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.646+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.647+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.647+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.647+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.647+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.648+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.648+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.648+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.648+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.649+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.649+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.649+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.649+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.650+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.650+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.651+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.651+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.651+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.651+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.652+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.652+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.652+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.652+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.654+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.654+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.654+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.654+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.655+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.655+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.655+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.655+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.656+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.656+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.656+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.656+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.657+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.657+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.657+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.657+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.658+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.658+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.658+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.658+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.659+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.659+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.659+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.659+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.660+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.660+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.660+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.660+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.661+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.661+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.661+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.661+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.662+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.662+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.662+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.662+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.663+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.663+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.663+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.663+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.664+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.664+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.664+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.664+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.665+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.665+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.665+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.665+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.666+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.666+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.666+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.666+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.667+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.667+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.667+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.667+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.668+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.668+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.668+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.668+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.701+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.701+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.701+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.701+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.702+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.702+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.702+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.702+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.703+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.703+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.703+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.703+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.704+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.704+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.704+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.704+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.706+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.706+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.706+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.706+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.707+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.707+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.707+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.707+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.708+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.708+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.708+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.709+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.709+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.709+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.710+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.710+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.710+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.710+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.711+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.711+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.711+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.711+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.712+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.712+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.712+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.712+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.713+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.713+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.713+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.713+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.714+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.714+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.714+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.714+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.715+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.715+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.715+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.715+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.716+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.716+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.716+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.716+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.717+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.717+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.717+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.717+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.719+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.719+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.719+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.719+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.720+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.720+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.720+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.720+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.721+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.721+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.723+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.723+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.723+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.723+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.724+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.724+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.724+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.724+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.725+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.725+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.725+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.725+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.726+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.726+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.726+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.726+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.727+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.727+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.727+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.727+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.728+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.728+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.728+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.728+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.729+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.729+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.729+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.729+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.730+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.730+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.730+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.730+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.731+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.731+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.731+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.731+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.732+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.732+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.732+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.732+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.733+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.733+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.733+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.733+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.734+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.734+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.734+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.734+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.735+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.735+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.736+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.736+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.736+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.736+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.737+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.737+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.737+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.737+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.738+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.738+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.738+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.738+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.739+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.739+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.739+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.739+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.740+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.740+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.740+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.740+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.742+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.742+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.742+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.742+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.743+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.743+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.743+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.743+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.744+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.744+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.744+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.744+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.745+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.745+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.745+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.745+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.746+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.746+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.746+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.746+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.747+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.747+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.747+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.747+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.749+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.749+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.749+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.749+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.750+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.750+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.750+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.750+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.751+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.751+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.698+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.698+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.698+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.698+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.719+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.719+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.728+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.728+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.735+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.735+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.735+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.735+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.738+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.738+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.738+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.738+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.669+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.669+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.669+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.669+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.670+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.670+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.670+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.670+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.671+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.671+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.671+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.671+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.672+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.672+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.672+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.672+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.673+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.673+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.673+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.673+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.674+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.674+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.675+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.675+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.675+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.675+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.676+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.676+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.676+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.676+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.677+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.677+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.677+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.677+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.678+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.678+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.678+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.678+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.679+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.679+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.679+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.679+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.680+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.680+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.680+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.680+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.681+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.681+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.681+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.681+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.682+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.682+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.682+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.682+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.683+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.683+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.683+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.683+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.684+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.684+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.684+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.684+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.685+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.685+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.685+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.685+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.686+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.686+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.686+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.686+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.687+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.687+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.687+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.687+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.688+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.688+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.688+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.688+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.689+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.689+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.689+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.689+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.690+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.690+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.690+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.690+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.691+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.691+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.691+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.691+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.692+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.692+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.692+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.692+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.693+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.693+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.693+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.693+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.694+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.694+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.694+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.694+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.695+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.695+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.695+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.695+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.696+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.696+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.696+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.696+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.697+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.697+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.697+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.697+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.698+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.698+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.698+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.698+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.699+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.699+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.699+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.699+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.700+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.700+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.700+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.700+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.701+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.701+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.701+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.701+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.701+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.702+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.702+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.702+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.702+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.702+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.703+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.703+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.703+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.703+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.704+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.704+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.704+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.704+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.705+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.705+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.705+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.705+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.706+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.706+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.706+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.706+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.707+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.707+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.707+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.707+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.708+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.708+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.708+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.708+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.709+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.709+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.709+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.709+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.710+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.710+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.710+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.710+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.711+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.711+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.711+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.711+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.712+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.712+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.712+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.712+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.713+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.713+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.714+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.714+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.715+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.715+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.715+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.715+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.716+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.716+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.751+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.751+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.752+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.752+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.752+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.752+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.753+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.753+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.753+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.753+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.754+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.754+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.754+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.754+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.755+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.755+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.755+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.755+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.756+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.756+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.756+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.756+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.757+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.757+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.757+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.757+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.758+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.758+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.758+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.758+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.759+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.759+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.759+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.759+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.760+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.760+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.761+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.761+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.761+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.761+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.762+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.762+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.762+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.762+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.763+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.763+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.763+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.763+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.764+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.764+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.764+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.764+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.765+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.765+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.765+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.765+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.766+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.766+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.766+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.766+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.767+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.767+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.767+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.767+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.768+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.768+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.768+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.768+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.769+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.769+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.769+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.769+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.770+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.770+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.770+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.770+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.772+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.772+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.772+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.772+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.773+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.773+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.773+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.773+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.774+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.774+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.774+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.774+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.775+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.775+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.775+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.775+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.776+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.776+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.776+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.776+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.776+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.776+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.777+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.777+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.778+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.778+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.778+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.778+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.780+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.780+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.781+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.781+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.782+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.782+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.783+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.783+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.783+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.783+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.786+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.786+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.787+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.787+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.788+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.788+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.788+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.788+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.789+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.789+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.789+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.789+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.790+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.790+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.790+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.790+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.791+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.791+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.791+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.791+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.792+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.792+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.793+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.793+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.793+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.793+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.794+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.794+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.794+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.794+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.795+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.795+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.795+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.795+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.796+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.796+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.796+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.796+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.797+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.797+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.797+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.797+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.798+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.798+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.798+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.798+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.799+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.799+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.799+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.799+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.801+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.801+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.801+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.801+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.802+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.802+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.802+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.802+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.803+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.803+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.803+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.754+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.754+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.769+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.769+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.769+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.769+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.770+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.770+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.770+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.770+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.777+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.777+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.778+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.778+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.780+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.780+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.780+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.780+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.782+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.782+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.782+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.790+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.790+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.791+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.791+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.795+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.716+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.716+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.717+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.717+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.717+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.717+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.718+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.718+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.718+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.718+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.719+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.719+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.719+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.719+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.720+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.720+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.720+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.720+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.721+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.721+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.721+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.721+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.722+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.722+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.722+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.722+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.723+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.723+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.723+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.723+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.724+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.724+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.724+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.724+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.725+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.725+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.725+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.725+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.726+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.726+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.726+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.726+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.727+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.727+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.727+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.727+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.728+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.728+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.728+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.728+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.729+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.729+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.730+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.730+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.730+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.730+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.731+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.731+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.731+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.731+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.732+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.732+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.732+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.732+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.733+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.733+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.733+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.733+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.734+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.734+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.734+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.734+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.737+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.737+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.737+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.737+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.738+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.738+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.738+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.738+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.739+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.739+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.739+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.739+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.740+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.740+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.740+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.740+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.741+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.741+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.741+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.741+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.742+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.742+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.742+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.742+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.743+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.743+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.743+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.743+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.744+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.744+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.744+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.744+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.745+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.745+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.745+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.745+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.746+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.746+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.746+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.746+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.747+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.747+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.747+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.747+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.748+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.748+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.748+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.748+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.749+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.749+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.749+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.749+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.750+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.750+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.750+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.750+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.750+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.750+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.751+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.751+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.751+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.751+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.753+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.753+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.753+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.753+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.754+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.754+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.754+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.754+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.755+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.755+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.755+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.755+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.756+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.756+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.756+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.756+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.758+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.758+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.758+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.758+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.759+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.759+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.759+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.759+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.760+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.760+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.760+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.760+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.761+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.761+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.761+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.761+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.762+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.762+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.762+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.762+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.763+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.763+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.763+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.763+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.764+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.764+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.764+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.803+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.803+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.803+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.804+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.804+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.804+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.804+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.805+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.805+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.805+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.805+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.806+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.806+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.806+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.806+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.807+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.807+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.807+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.807+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.808+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.808+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.808+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.808+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.809+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.809+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.809+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.809+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.810+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.810+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.810+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.810+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.811+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.811+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.811+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.811+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.812+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.812+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.812+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.812+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.813+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.813+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.813+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.813+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.814+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.814+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.814+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.814+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.815+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.815+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.815+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.815+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.816+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.816+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.816+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.816+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.817+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.817+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.817+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.817+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.818+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.818+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.818+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.818+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.819+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.819+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.819+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.819+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.820+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.820+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.820+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.820+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.821+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.821+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.821+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.821+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.822+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.822+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.822+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.822+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.823+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.823+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.823+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.823+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.824+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.824+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.824+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.824+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.825+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.825+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.825+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.825+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.826+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.826+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.826+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.826+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.827+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.827+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.827+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.827+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.828+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.828+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.828+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.828+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.829+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.829+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.829+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.829+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.830+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.830+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.830+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.830+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.832+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.832+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.832+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.832+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.833+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.833+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.833+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.833+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.834+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.834+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.834+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.834+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.835+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.835+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.835+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.835+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.836+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.836+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.836+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.836+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.837+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.837+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.837+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.837+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.838+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.838+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.838+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.838+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.839+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.839+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.839+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.839+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.840+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.840+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.840+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.840+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.841+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.841+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.842+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.842+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.842+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.842+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.843+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.843+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.843+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.843+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.844+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.844+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.844+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.844+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.845+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.845+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.846+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.846+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.846+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.846+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.847+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.847+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.847+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.847+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.848+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.848+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.848+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.848+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.849+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.849+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.849+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.849+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.850+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.795+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.795+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.795+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.796+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.796+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.801+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.801+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.824+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.824+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.826+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.826+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.836+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.836+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.836+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.836+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.764+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.765+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.765+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.765+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.765+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.766+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.766+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.766+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.766+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.767+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.767+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.767+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.767+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.768+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.768+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.768+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.768+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.769+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.769+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.769+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.769+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.770+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.770+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.770+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.770+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.771+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.771+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.771+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.771+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.772+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.772+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.772+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.772+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.773+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.773+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.773+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.773+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.774+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.774+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.774+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.774+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.775+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.775+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.775+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.775+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.776+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.776+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.776+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.776+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.777+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.777+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.777+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.777+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.778+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.778+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.778+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.778+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.779+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.779+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.779+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.779+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.780+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.780+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.782+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.782+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.782+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.782+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.783+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.783+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.783+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.783+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.784+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.784+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.784+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.784+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.785+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.785+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.785+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.785+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.786+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.786+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.786+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.786+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.787+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.787+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.788+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.788+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.788+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.788+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.789+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.789+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.789+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.789+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.790+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.790+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.790+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.790+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.791+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.791+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.791+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.791+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.792+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.792+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.792+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.792+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.793+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.793+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.793+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.793+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.794+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.794+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.794+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.794+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.795+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.795+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.795+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.795+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.796+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.796+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.796+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.796+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.797+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.797+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.797+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.797+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.798+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.798+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.798+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.798+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.799+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.799+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.799+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.799+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.800+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.800+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.800+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.800+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.801+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.801+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.801+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.801+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.802+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.802+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.802+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.802+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.803+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.803+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.803+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.803+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.804+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.804+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.804+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.804+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.805+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.805+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.805+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.805+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.806+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.806+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.806+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.806+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.807+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.807+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.808+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.808+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.808+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.808+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.809+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.809+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.809+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.809+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.811+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.811+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.811+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.811+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.812+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.812+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.812+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.850+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.850+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.850+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.851+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.851+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.851+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.851+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.852+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.852+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.852+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.852+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.853+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.853+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.853+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.853+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.854+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.854+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.854+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.854+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.855+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.855+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.855+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.855+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.856+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.856+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.857+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.857+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.857+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.857+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.858+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.858+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.858+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.858+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.859+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.859+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.859+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.859+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.860+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.860+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.860+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.860+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.861+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.861+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.861+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.861+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.862+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.862+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.862+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.862+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.863+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.863+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.863+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.863+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.864+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.864+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.864+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.864+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.865+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.865+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.865+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.865+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.866+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.866+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.866+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.866+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.867+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.867+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.867+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.867+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.868+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.868+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.868+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.868+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.869+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.869+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.869+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.869+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.870+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.870+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.870+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.870+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.871+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.871+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.872+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.872+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.872+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.872+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.873+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.873+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.873+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.873+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.874+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.874+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.874+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.874+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.875+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.875+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.875+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.875+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.876+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.876+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.876+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.876+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.877+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.877+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.877+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.877+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.878+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.878+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.878+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.878+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.879+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.879+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.879+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.879+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.880+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.880+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.880+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.880+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.881+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.881+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.881+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.881+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.882+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.882+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.882+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.882+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.883+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.883+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.883+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.883+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.884+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.884+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.884+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.884+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.885+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.885+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.885+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.885+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.886+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.886+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.886+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.886+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.887+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.887+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.887+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.887+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.888+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.888+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.888+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.888+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.889+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.889+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.889+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.889+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.890+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.890+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.890+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.890+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.891+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.891+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.891+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.891+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.892+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.892+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.892+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.892+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.893+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.893+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.893+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.893+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.894+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.894+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.894+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.894+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.895+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.895+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.895+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.895+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.896+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.896+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.896+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.896+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.897+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.897+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.897+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.897+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.898+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.843+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.843+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.843+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.843+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.844+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.844+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.850+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.850+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.856+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.856+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.886+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.886+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.886+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.886+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.812+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.813+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.813+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.813+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.813+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.814+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.814+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.814+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.814+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.815+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.815+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.815+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.815+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.816+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.816+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.816+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.816+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.817+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.817+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.817+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.817+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.819+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.819+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.819+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.819+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.820+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.820+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.820+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.820+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.821+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.821+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.821+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.821+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.822+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.822+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.822+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.823+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.823+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.823+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.824+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.824+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.824+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.824+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.825+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.825+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.825+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.825+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.826+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.826+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.826+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.826+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.827+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.827+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.827+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.827+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.828+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.828+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.828+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.828+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.829+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.829+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.829+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.829+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.830+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.830+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.830+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.830+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.831+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.831+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.831+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.831+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.832+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.832+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.832+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.832+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.833+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.833+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.833+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.833+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.834+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.834+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.834+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.834+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.835+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.835+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.835+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.835+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.836+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.836+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.836+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.836+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.838+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.838+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.838+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.838+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.839+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.839+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.839+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.839+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.840+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.840+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.840+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.840+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.841+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.841+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.841+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.841+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.842+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.842+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.842+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.842+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.843+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.843+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.843+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.843+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.844+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.844+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.844+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.844+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.845+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.845+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.845+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.845+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.846+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.846+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.846+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.846+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.847+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.847+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.847+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.847+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.848+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.848+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.848+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.848+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.849+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.849+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.849+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.849+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.850+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.850+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.850+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.850+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.851+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.851+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.851+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.851+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.852+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.852+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.852+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.852+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.853+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.853+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.853+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.853+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.854+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.854+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.854+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.854+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.855+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.855+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.855+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.855+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.856+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.856+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.856+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.856+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.857+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.857+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.857+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.857+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.859+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.859+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.859+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.859+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.860+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.898+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.898+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.898+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.899+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.899+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.899+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.899+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.900+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.900+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.900+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.900+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.901+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.901+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.901+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.901+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.902+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.902+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.902+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.902+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.903+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.903+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.904+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.904+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.904+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.904+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.905+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.905+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.905+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.905+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.906+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.906+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.906+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.906+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.907+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.907+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.907+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.907+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.908+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.908+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.908+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.908+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.909+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.909+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.909+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.909+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.910+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.910+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.910+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.910+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.910+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.910+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.911+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.911+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.913+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.913+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.913+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.913+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.914+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.914+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.914+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.914+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.916+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.916+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.916+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.916+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.917+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.917+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.917+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.917+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.918+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.918+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.918+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.918+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.919+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.919+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.919+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.919+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.920+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.920+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.920+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.920+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.921+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.921+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.921+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.921+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.922+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.922+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.922+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.922+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.923+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.923+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.923+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.923+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.924+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.924+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.924+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.924+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.925+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.925+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.926+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.926+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.926+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.926+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.927+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.927+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.927+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.927+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.928+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.928+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.928+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.928+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.929+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.929+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.929+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.929+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.930+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.930+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.930+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.930+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.932+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.932+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.932+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.932+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.933+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.933+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.933+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.933+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.934+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.934+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.934+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.934+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.935+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.935+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.935+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.935+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.936+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.936+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.936+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.936+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.936+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.936+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.937+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.937+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.937+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.937+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.938+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.938+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.938+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.938+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.941+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.941+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.941+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.941+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.942+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.942+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.942+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.942+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.943+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.943+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.943+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.943+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.944+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.944+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.944+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.944+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.945+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.945+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.945+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.945+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.946+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.946+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.946+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.946+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.947+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.947+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.924+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.924+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.927+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.927+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.927+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.927+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.928+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.860+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.860+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.860+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.861+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.861+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.861+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.861+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.862+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.862+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.862+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.862+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.863+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.863+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.863+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.863+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.864+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.864+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.864+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.864+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.865+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.865+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.865+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.865+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.867+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.867+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.867+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.867+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.868+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.868+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.868+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.868+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.869+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.869+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.869+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.869+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.870+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.870+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.870+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.870+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.871+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.871+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.871+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.871+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.872+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.872+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.872+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.872+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.873+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.873+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.873+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.873+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.874+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.874+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.874+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.874+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.875+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.875+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.875+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.875+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.876+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.876+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.876+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.876+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.877+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.877+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.877+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.877+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.878+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.878+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.878+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.878+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.879+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.879+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.879+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.879+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.880+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.880+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.880+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.880+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.881+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.881+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.881+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.881+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.882+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.882+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.883+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.883+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.883+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.883+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.885+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.885+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.885+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.885+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.886+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.886+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.886+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.886+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.887+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.887+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.887+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.887+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.888+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.888+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.888+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.888+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.889+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.889+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.889+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.889+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.890+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.890+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.890+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.890+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.891+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.891+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.891+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.891+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.892+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.892+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.892+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.892+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.894+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.894+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.895+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.895+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.895+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.895+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.896+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.896+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.896+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.896+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.897+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.897+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.897+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.897+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.898+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.898+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.898+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.898+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.899+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.899+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.899+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.899+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.900+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.900+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.900+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.900+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.901+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.901+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.901+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.901+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.902+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.902+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.902+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.902+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.903+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.903+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.904+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.904+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.904+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.904+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.905+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.905+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.905+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.905+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.906+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.906+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.906+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.906+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.906+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.907+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.907+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.907+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.907+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.907+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.908+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.908+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.908+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.908+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.947+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.947+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.948+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.948+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.948+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.948+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.949+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.949+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.949+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.949+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.950+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.950+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.950+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.950+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.951+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.951+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.951+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.951+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.952+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.952+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.952+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.952+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.953+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.953+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.953+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.953+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.954+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.954+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.954+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.954+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.955+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.955+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.955+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.955+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.956+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.956+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.956+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.956+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.957+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.957+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.957+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.957+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.958+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.958+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.958+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.958+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.959+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.959+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.959+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.959+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.960+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.960+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.960+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.960+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.961+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.961+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.961+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.961+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.962+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.962+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.962+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.962+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.963+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.963+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.963+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.963+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.964+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.964+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.964+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.964+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.965+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.965+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.965+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.965+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.966+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.966+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.966+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.966+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.967+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.967+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.968+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.968+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.968+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.968+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.969+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.969+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.969+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.969+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.970+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.970+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.970+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.970+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.971+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.971+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.971+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.971+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.972+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.972+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.972+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.972+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.973+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.973+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.973+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.973+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.974+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.974+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.974+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.974+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.975+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.975+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.975+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.975+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.976+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.976+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.976+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.976+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.977+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.977+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.977+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.977+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.978+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.978+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.979+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.979+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.979+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.979+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.980+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.980+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.980+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.980+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.981+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.981+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.981+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.981+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.982+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.982+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.982+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.982+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.984+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.984+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.984+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.984+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.984+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.985+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.985+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.985+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.986+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.986+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.986+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.986+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.987+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.987+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.987+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.987+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.988+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.988+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.988+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.988+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.989+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.989+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.989+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.989+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.990+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.990+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.990+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.990+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.991+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.991+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.991+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.991+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.992+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.992+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.992+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.992+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.993+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.993+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.993+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.993+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.994+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.994+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.994+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.994+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.995+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.995+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.938+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.938+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.940+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.940+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.948+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.948+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.948+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.948+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.952+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.952+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.955+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.955+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.967+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.968+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.968+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.969+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.969+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.971+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.971+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.971+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.971+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.974+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.974+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.974+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.974+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.975+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.975+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.979+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.979+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.979+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.979+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.909+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.909+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.909+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.909+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.910+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.910+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.910+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.910+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.911+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.911+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.911+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.911+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.912+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.912+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.912+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.912+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.913+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.913+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.913+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.913+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.914+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.914+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.914+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.914+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.915+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.915+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.915+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.915+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.916+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.916+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.916+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.916+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.917+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.917+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.917+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.917+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.918+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.918+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.918+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.918+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.919+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.919+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.919+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.919+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.920+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.920+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.920+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.920+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.921+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.921+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.921+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.921+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.922+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.922+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.922+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.922+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.923+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.923+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.923+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.923+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.924+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.924+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.924+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.924+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.925+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.925+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.925+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.925+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.926+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.926+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.926+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.926+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.927+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.927+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.929+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.929+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.929+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.929+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.930+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.930+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.930+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.930+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.930+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.931+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.931+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.931+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.932+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.932+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.932+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.932+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.933+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.933+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.933+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.933+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.933+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.933+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.934+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.934+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.934+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.934+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.935+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.935+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.935+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.935+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.936+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.936+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.936+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.936+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.937+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.937+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.937+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.937+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.938+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.938+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.938+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.938+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.939+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.939+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.939+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.939+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.940+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.940+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.940+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.940+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.941+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.941+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.941+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.941+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.942+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.942+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.942+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.942+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.943+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.943+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.943+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.943+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.944+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.944+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.944+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.944+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.945+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.945+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.945+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.945+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.946+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.946+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.946+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.946+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.947+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.947+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.947+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.947+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.948+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.948+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.948+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.948+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.949+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.949+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.949+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.949+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.950+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.950+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.950+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.950+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.951+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.951+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.951+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.951+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.952+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.952+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.952+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.952+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.953+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.953+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.953+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.953+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.954+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.954+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.954+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.954+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.955+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.955+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.955+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.955+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.956+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.956+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.956+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.956+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.995+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.995+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.996+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.996+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.996+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.996+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.997+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.997+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.997+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.997+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.998+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.998+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.998+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.998+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.999+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.999+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.999+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:00.999+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.000+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.000+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.000+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.000+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.001+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.001+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.001+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.001+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.002+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.002+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.002+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.002+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.003+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.003+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.003+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.003+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.004+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.004+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.004+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.004+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.005+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.005+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.005+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.005+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.006+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.006+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.006+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.006+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.007+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.007+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.007+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.007+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.008+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.008+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.008+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.008+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.009+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.009+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.009+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.009+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.010+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.010+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.010+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.010+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.011+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.011+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.011+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.011+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.012+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.012+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.012+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.012+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.014+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.014+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.014+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.014+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.015+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.015+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.015+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.015+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.016+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.016+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.016+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.016+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.017+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.017+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.018+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.018+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.018+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.018+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.020+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.020+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.020+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.020+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.021+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.021+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.021+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.021+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.022+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.022+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.022+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.022+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.023+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.023+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.023+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.023+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.024+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.024+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.024+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.024+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.025+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.025+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.025+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.025+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.026+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.026+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.028+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.028+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.028+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.028+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.029+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.029+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.029+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.029+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.030+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.030+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.030+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.030+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.031+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.031+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.031+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.031+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.032+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.032+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.032+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.032+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.033+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.033+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.033+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.033+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.034+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.034+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.034+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.034+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.035+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.035+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.035+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.035+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.036+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.036+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.036+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.036+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.037+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.037+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.037+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.037+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.038+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.038+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.038+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.038+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.039+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.039+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.039+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.039+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.040+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.040+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.040+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.040+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.041+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.041+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.041+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.041+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.042+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.042+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.042+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.042+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.043+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.043+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.043+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.043+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.986+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.986+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.987+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.987+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.990+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.990+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.990+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.990+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.998+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.998+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.998+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.998+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:00.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.015+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.015+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.019+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.019+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.030+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.030+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.030+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.030+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.031+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.031+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.031+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.957+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.957+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.957+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.957+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.958+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.958+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.958+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.958+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.959+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.959+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.959+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.959+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.961+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.961+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.961+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.961+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.962+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.962+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.962+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.962+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.963+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.963+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.963+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.963+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.964+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.964+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.964+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.964+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.965+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.965+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.965+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.965+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.966+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.966+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.966+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.966+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.967+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.967+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.967+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.967+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.968+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.968+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.968+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.968+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.969+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.969+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.969+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.969+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.970+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.970+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.970+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.970+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.971+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.971+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.971+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.971+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.972+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.972+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.972+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.972+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.973+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.973+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.973+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.973+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.974+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.974+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.974+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.974+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.975+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.975+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.975+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.975+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.976+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.976+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.977+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.977+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.977+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.977+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.978+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.978+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.978+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.978+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.979+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.979+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.979+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.979+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.980+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.980+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.980+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.980+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.981+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.981+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.981+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.981+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.982+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.982+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.982+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.982+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.983+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.983+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.983+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.983+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.984+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.984+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.984+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.984+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.985+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.985+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.985+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.985+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.986+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.986+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.986+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.986+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.987+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.987+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.987+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.987+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.988+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.988+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.988+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.988+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.989+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.989+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.989+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.989+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.989+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.989+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.990+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.990+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.990+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.990+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.991+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.991+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.991+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.991+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.992+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.992+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.993+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.993+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.993+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.993+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.995+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.995+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.997+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.997+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.997+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.997+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.998+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.998+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.998+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.998+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.998+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.998+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.999+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.999+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.999+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:00.999+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.000+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.000+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.001+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.001+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.001+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.001+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.002+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.002+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.002+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.002+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.003+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.003+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.003+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.003+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.004+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.004+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.004+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.004+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.005+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.005+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.044+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.044+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.044+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.044+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.045+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.045+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.045+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.045+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.046+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.046+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.046+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.046+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.047+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.047+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.047+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.047+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.048+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.048+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.048+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.048+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.049+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.049+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.049+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.049+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.051+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.051+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.051+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.051+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.051+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.051+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.052+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.052+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.052+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.052+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.053+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.053+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.053+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.053+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.055+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.055+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.056+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.056+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.056+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.056+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.057+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.057+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.057+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.057+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.058+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.058+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.058+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.058+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.061+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.061+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.061+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.061+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.062+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.062+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.062+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.062+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.063+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.063+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.063+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.063+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.064+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.064+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.064+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.064+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.065+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.065+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.065+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.065+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.066+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.066+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.066+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.066+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.067+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.067+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.067+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.067+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.068+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.068+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.068+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.068+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.069+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.069+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.069+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.069+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.070+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.070+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.070+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.070+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.071+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.071+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.071+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.071+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.072+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.072+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.072+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.072+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.073+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.073+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.073+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.073+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.074+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.074+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.074+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.074+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.076+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.076+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.076+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.076+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.077+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.077+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.077+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.077+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.078+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.078+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.078+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.079+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.079+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.079+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.080+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.080+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.080+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.080+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.081+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.081+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.081+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.081+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.082+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.082+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.082+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.082+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.083+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.083+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.083+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.083+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.084+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.084+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.084+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.084+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.085+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.085+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.085+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.085+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.086+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.086+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.086+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.086+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.087+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.087+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.089+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.089+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.089+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.089+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.090+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.090+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.090+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.090+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.091+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.091+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.091+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.091+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.092+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.092+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.092+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.092+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.093+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.093+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.093+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.093+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.094+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.094+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.094+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.037+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.037+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.038+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.038+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.039+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.039+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.040+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.040+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.045+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.045+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.046+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.046+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.046+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.046+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.047+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.047+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.050+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.050+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.051+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.051+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.054+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.054+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.056+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.056+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.057+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.057+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.067+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.067+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.068+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.070+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.070+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.072+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.072+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.072+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.072+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.080+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.080+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.081+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.081+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.005+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.005+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.006+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.006+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.006+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.006+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.007+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.007+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.007+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.007+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.008+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.008+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.008+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.008+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.009+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.009+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.009+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.009+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.010+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.010+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.010+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.010+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.012+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.012+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.012+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.012+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.012+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.013+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.013+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.013+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.013+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.013+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.014+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.014+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.014+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.014+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.015+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.015+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.015+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.015+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.016+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.016+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.016+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.016+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.017+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.017+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.017+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.017+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.018+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.018+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.018+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.018+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.019+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.019+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.019+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.019+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.021+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.021+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.021+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.021+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.022+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.022+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.022+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.022+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.023+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.023+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.023+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.023+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.024+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.024+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.024+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.024+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.025+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.025+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.025+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.025+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.026+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.026+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.026+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.026+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.027+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.027+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.028+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.028+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.028+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.028+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.029+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.029+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.029+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.029+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.030+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.030+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.030+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.030+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.030+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.030+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.031+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.031+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.031+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.031+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.032+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.032+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.032+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.032+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.033+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.033+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.033+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.033+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.034+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.034+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.035+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.035+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.035+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.035+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.036+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.036+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.036+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.036+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.037+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.037+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.037+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.037+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.038+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.038+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.038+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.038+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.039+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.039+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.039+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.039+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.040+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.040+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.040+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.040+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.042+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.042+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.043+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.043+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.043+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.043+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.044+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.044+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.044+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.044+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.045+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.045+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.045+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.045+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.046+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.046+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.046+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.046+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.047+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.047+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.047+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.047+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.049+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.049+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.049+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.049+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.050+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.050+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.050+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.050+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.051+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.051+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.051+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.051+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.052+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.094+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.095+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.095+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.095+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.095+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.097+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.097+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.097+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.097+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.098+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.098+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.098+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.098+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.099+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.099+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.099+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.099+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.101+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.101+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.103+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.103+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.103+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.103+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.104+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.104+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.104+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.104+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.105+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.105+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.105+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.105+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.106+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.106+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.106+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.106+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.107+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.107+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.107+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.107+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.108+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.108+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.108+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.108+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.109+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.109+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.109+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.109+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.110+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.110+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.110+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.110+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.111+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.111+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.111+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.111+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.112+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.112+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.112+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.112+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.113+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.113+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.113+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.113+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.114+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.114+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.114+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.114+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.115+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.115+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.115+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.115+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.116+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.116+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.116+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.116+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.117+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.117+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.117+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.117+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.120+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.120+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.120+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.120+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.121+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.121+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.121+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.121+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.122+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.122+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.124+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.124+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.124+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.124+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.125+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.125+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.125+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.125+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.126+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.126+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.126+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.126+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.127+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.127+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.127+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.127+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.127+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.127+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.128+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.128+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.128+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.128+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.129+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.129+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.129+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.129+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.130+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.130+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.130+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.130+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.131+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.131+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.131+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.131+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.132+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.132+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.132+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.132+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.133+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.133+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.133+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.133+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.134+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.134+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.134+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.134+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.135+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.135+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.135+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.135+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.136+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.136+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.136+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.136+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.137+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.137+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.137+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.137+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.139+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.139+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.139+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.139+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.139+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.139+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.140+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.140+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.140+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.140+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.141+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.141+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.141+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.083+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.083+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.087+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.087+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.087+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.093+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.093+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.093+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.093+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.116+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.116+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.117+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.117+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.052+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.052+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.052+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.053+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.053+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.053+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.053+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.055+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.055+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.056+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.056+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.056+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.056+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.057+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.057+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.057+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.057+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.058+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.058+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.058+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.058+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.059+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.059+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.059+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.059+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.060+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.060+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.060+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.060+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.061+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.061+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.061+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.061+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.062+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.062+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.062+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.062+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.063+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.063+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.063+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.063+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.064+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.064+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.064+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.064+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.065+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.065+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.065+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.065+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.066+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.066+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.066+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.066+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.067+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.067+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.067+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.067+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.068+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.068+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.068+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.068+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.069+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.069+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.069+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.069+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.070+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.070+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.070+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.070+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.071+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.071+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.071+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.071+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.072+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.072+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.072+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.072+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.073+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.073+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.073+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.073+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.074+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.074+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.074+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.074+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.075+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.075+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.075+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.075+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.076+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.076+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.076+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.076+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.077+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.077+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.077+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.077+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.078+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.078+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.078+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.078+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.079+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.079+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.079+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.079+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.080+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.080+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.080+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.080+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.081+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.081+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.081+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.082+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.082+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.082+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.083+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.083+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.083+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.083+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.084+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.084+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.084+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.084+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.085+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.085+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.085+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.085+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.086+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.086+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.086+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.086+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.087+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.087+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.087+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.087+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.088+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.088+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.088+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.088+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.089+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.089+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.090+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.090+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.090+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.090+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.091+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.091+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.091+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.091+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.092+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.092+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.092+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.092+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.093+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.093+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.093+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.093+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.094+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.094+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.094+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.094+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.095+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.095+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.095+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.095+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.096+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.096+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.096+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.096+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.097+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.097+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.097+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.097+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.098+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.098+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.098+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.098+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.099+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.099+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.099+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.141+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.142+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.142+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.142+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.142+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.143+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.143+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.143+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.143+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.144+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.144+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.144+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.144+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.145+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.145+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.145+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.145+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.146+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.146+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.146+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.146+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.147+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.147+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.147+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.147+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.148+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.148+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.148+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.148+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.149+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.149+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.149+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.149+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.150+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.150+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.150+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.150+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.151+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.151+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.151+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.151+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.152+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.152+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.152+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.152+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.153+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.153+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.153+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.153+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.154+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.154+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.154+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.154+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.155+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.155+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.155+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.155+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.156+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.156+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.156+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.156+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.157+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.157+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.157+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.157+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.158+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.158+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.158+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.158+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.159+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.159+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.159+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.159+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.160+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.160+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.160+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.160+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.163+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.163+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.163+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.163+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.164+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.164+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.164+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.164+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.165+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.165+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.165+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.165+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.168+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.168+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.168+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.168+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.169+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.169+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.169+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.169+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.170+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.170+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.170+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.170+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.171+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.171+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.171+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.171+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.172+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.172+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.172+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.172+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.173+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.173+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.173+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.173+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.174+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.174+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.174+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.174+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.175+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.175+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.175+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.175+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.176+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.176+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.176+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.176+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.177+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.177+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.177+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.177+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.178+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.178+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.178+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.178+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.180+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.180+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.180+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.180+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.181+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.181+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.181+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.181+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.182+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.182+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.182+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.182+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.183+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.183+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.184+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.184+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.184+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.184+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.186+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.186+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.186+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.186+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.187+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.187+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.187+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.187+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.133+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.133+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.137+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.137+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.142+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.142+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.147+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.147+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.153+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.153+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.153+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.153+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.154+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.154+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.159+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.159+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.163+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.163+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.164+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.164+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.168+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.168+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.174+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.174+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.174+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.174+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.099+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.100+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.100+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.100+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.100+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.101+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.101+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.101+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.101+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.102+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.102+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.102+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.102+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.103+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.103+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.103+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.103+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.104+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.104+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.104+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.104+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.105+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.105+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.105+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.105+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.106+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.106+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.106+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.106+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.107+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.107+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.107+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.107+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.108+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.108+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.108+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.108+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.110+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.110+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.110+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.110+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.111+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.111+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.111+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.111+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.112+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.112+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.112+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.112+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.113+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.113+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.113+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.113+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.114+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.114+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.114+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.114+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.115+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.115+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.115+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.115+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.116+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.116+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.116+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.116+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.117+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.117+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.117+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.117+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.118+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.118+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.118+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.118+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.119+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.119+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.119+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.119+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.120+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.120+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.120+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.120+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.121+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.121+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.121+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.121+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.121+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.121+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.122+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.122+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.122+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.122+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.123+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.123+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.123+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.123+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.124+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.124+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.124+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.124+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.125+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.125+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.125+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.125+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.126+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.126+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.126+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.126+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.127+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.127+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.127+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.127+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.128+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.128+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.128+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.128+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.129+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.129+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.129+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.129+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.130+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.130+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.130+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.130+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.131+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.131+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.131+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.131+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.132+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.132+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.132+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.132+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.133+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.133+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.133+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.133+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.134+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.134+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.134+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.134+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.135+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.135+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.135+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.135+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.136+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.136+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.136+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.136+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.137+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.137+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.137+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.137+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.139+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.139+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.139+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.139+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.140+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.140+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.140+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.140+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.141+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.141+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.141+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.141+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.142+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.142+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.142+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.142+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.143+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.143+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.143+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.143+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.144+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.144+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.144+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.144+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.145+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.145+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.145+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.189+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.189+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.189+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.189+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.190+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.190+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.191+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.191+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.191+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.191+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.192+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.192+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.192+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.192+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.193+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.193+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.193+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.193+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.194+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.194+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.178+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.178+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.178+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.178+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.179+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.179+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.185+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.185+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.145+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.146+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.146+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.146+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.146+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.146+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.146+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.147+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.147+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.147+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.147+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.148+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.148+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.148+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.148+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.149+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.149+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.149+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.149+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.150+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.150+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.150+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.150+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.151+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.151+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.151+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.151+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.152+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.152+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.152+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.152+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.153+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.153+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.153+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.153+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.154+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.154+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.154+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.154+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.155+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.155+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.155+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.155+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.157+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.157+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.158+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.158+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.158+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.158+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.159+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.159+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.159+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.159+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.160+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.160+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.160+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.160+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.161+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.161+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.161+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.161+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.162+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.162+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.162+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.162+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.163+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.163+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.163+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.163+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.164+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.164+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.164+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.164+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.165+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.165+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.165+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.165+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.166+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.166+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.166+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.166+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.167+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.167+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.167+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.167+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.168+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.168+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.168+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.168+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.169+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.169+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.169+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.169+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.170+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.170+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.170+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.170+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.171+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.171+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.171+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.171+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.172+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.172+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.172+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.172+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.173+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.173+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.173+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.173+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.174+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.174+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.174+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.174+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.176+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.176+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.176+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.176+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.177+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.177+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.177+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.177+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.178+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.178+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.178+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.178+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.179+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.179+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.179+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.179+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.180+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.180+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.180+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.180+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.181+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.181+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.181+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.182+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.182+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.184+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.184+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.184+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.184+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.184+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.184+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.185+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.185+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.185+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.185+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.186+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.186+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.186+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.186+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.187+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.187+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.187+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.187+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.189+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.189+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.189+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.189+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.190+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.190+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.190+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.190+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.191+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.191+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.191+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.191+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.192+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.192+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.194+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.194+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.195+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.195+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.195+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.195+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.196+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.196+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.196+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.196+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.197+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.197+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.197+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.197+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.198+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.198+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.198+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.198+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.199+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.199+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.199+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.199+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.200+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.200+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.200+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.200+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.202+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.202+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.202+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.202+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.203+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.203+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.203+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.203+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.204+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.204+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.204+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.204+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.205+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.205+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.205+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.205+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.206+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.206+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.206+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.206+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.207+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.207+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.207+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.207+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.208+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.208+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.208+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.208+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.209+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.209+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.209+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.209+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.210+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.210+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.210+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.210+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.211+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.211+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.211+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.211+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.212+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.212+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.212+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.212+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.213+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.213+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.213+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.213+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.213+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.213+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.214+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.214+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.214+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.214+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.215+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.215+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.215+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.215+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.216+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.216+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.216+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.216+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.217+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.217+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.217+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.217+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.218+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.218+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.218+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.218+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.219+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.219+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.219+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.219+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.220+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.220+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.222+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.222+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.222+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.222+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.223+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.223+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.223+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.223+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.224+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.224+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.224+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.224+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.225+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.225+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.225+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.225+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.226+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.226+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.226+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.226+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.227+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.227+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.227+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.227+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.228+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.228+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.228+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.228+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.229+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.229+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.229+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.229+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.230+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.230+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.230+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.230+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.231+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.231+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.231+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.231+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.232+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.232+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.232+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.232+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.233+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.233+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.233+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.233+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.233+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.233+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.234+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.234+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.234+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.234+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.235+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.235+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.235+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.235+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.236+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.236+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.236+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.236+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.237+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.237+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.237+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.237+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.238+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.238+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.240+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.240+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.240+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.240+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.198+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.198+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.198+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.198+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.205+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.205+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.205+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.205+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.209+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.209+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.210+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.210+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.211+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.211+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.218+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.218+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.220+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.220+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.222+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.222+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.236+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.236+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.236+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.236+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.193+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.193+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.193+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.193+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.194+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.194+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.194+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.194+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.195+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.195+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.195+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.195+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.196+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.196+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.197+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.197+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.197+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.197+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.198+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.198+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.198+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.198+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.198+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.198+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.199+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.199+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.199+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.199+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.200+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.200+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.200+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.200+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.201+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.201+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.201+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.201+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.202+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.202+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.202+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.202+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.203+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.203+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.203+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.203+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.204+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.204+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.204+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.204+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.205+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.205+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.205+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.205+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.206+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.206+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.206+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.206+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.207+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.207+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.207+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.207+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.207+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.207+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.208+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.208+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.208+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.208+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.209+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.209+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.209+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.209+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.210+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.210+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.210+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.210+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.211+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.211+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.211+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.211+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.212+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.212+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.212+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.212+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.213+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.213+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.213+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.213+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.214+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.214+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.214+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.214+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.215+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.215+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.215+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.215+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.216+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.216+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.216+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.216+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.217+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.217+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.217+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.217+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.218+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.218+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.218+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.218+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.219+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.219+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.219+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.219+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.219+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.219+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.220+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.220+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.220+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.220+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.221+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.221+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.222+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.222+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.222+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.222+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.223+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.223+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.223+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.223+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.223+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.223+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.224+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.224+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.224+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.224+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.225+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.225+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.225+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.225+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.227+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.227+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.227+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.227+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.228+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.228+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.228+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.228+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.229+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.229+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.229+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.229+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.230+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.230+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.230+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.230+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.231+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.231+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.231+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.231+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.232+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.232+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.232+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.232+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.232+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.233+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.233+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.233+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.233+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.233+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.234+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.234+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.234+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.234+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.235+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.235+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.235+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.235+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.236+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.236+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.236+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.236+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.237+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.237+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.241+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.241+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.241+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.241+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.242+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.242+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.242+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.242+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.243+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.243+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.243+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.243+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.244+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.244+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.244+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.244+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.245+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.245+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.245+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.245+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.246+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.246+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.246+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.246+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.247+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.247+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.247+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.247+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.248+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.248+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.248+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.248+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.249+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.249+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.249+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.249+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.250+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.250+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.250+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.250+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.241+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.241+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.241+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.242+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.237+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.237+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.238+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.238+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.238+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.238+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.239+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.239+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.239+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.239+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.240+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.240+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.240+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.240+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.241+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.241+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.241+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.241+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.242+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.242+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.242+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.242+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.243+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.243+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.243+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.243+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.244+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.244+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.244+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.244+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.244+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.244+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.245+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.245+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.246+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.246+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.246+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.246+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.247+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.247+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.247+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.247+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.248+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.248+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.248+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.248+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.249+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.249+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.249+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.249+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.250+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.250+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.250+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.250+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.250+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.250+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.250+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.251+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.251+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.251+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.251+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.251+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.251+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.251+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.251+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.251+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.252+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.252+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.252+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.252+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.252+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.252+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.252+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.252+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.253+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.253+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.253+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.253+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.253+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.253+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.254+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.254+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.254+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.254+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.255+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.255+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.255+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.255+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.256+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.256+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.256+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.256+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.257+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.257+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.257+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.257+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.258+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.258+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.258+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.258+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.259+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.259+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.259+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.259+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.260+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.260+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.260+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.260+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.261+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.261+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.261+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.261+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.262+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.262+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.262+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.262+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.263+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.263+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.263+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.263+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.265+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.265+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.265+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.265+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.266+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.266+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.267+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.267+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.267+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.267+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.269+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.269+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.269+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.269+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.270+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.270+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.270+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.270+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.270+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.271+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.271+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.271+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.272+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.272+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.274+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.274+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.274+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.274+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.274+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.275+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.276+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.276+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.276+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.277+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.277+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.277+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.277+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.278+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.278+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.278+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.278+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.279+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.279+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.279+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.279+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.280+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.280+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.280+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.280+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.281+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.281+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.281+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.281+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.282+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.282+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.282+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.282+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.283+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.283+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.283+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.283+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.284+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.284+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.284+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.284+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.285+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.285+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.285+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.285+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.286+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.286+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.287+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.287+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.287+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.287+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.287+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.288+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.288+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.288+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.289+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.289+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.289+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.289+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.290+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.290+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.290+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.290+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.291+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.291+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.291+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.291+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.293+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.293+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.294+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.294+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.295+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.295+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.295+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.295+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.296+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.296+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.296+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.296+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.297+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.297+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.297+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.297+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.298+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.298+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.298+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.298+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.299+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.299+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.299+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.299+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.300+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.300+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.300+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.300+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.301+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.301+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.301+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.301+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.302+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.302+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.302+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.302+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.303+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.303+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.303+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.303+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.304+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.304+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.305+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.305+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.305+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.305+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.255+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.255+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.269+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.269+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.287+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.287+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.288+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.288+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.288+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.288+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.291+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.291+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.295+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.295+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.295+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.295+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.296+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.296+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.301+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.301+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.304+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.304+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.254+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.254+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.254+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.254+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.255+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.255+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.255+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.255+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.256+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.256+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.256+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.256+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.257+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.257+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.257+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.257+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.257+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.257+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.258+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.258+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.258+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.258+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.259+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.259+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.259+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.259+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.260+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.260+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.260+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.260+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.261+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.261+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.261+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.261+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.262+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.262+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.262+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.262+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.263+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.263+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.263+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.263+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.264+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.264+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.264+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.264+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.265+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.265+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.265+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.265+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.266+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.266+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.266+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.266+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.267+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.267+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.267+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.267+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.268+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.268+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.268+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.268+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.269+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.269+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.269+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.269+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.269+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.269+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.270+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.270+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.270+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.270+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.271+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.271+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.271+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.271+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.272+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.272+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.272+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.272+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.273+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.273+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.273+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.273+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.274+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.274+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.274+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.274+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.275+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.275+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.275+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.275+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.276+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.276+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.276+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.276+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.277+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.277+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.277+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.277+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.278+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.278+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.278+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.278+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.279+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.279+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.279+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.279+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.280+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.280+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.280+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.280+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.282+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.282+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.282+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.282+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.283+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.283+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.283+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.283+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.284+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.284+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.284+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.284+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.285+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.285+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.285+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.285+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.286+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.286+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.286+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.286+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.287+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.287+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.287+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.287+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.288+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.288+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.288+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.288+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.289+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.289+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.289+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.289+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.290+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.290+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.290+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.290+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.291+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.291+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.291+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.291+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.294+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.294+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.294+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.294+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.295+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.295+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.295+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.295+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.296+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.296+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.296+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.296+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.297+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.297+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.297+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.297+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.298+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.298+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.298+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.298+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.299+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.299+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.299+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.299+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.300+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.300+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.300+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.300+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.301+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.301+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.301+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.301+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.306+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.306+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.306+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.306+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.307+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.307+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.307+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.307+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.308+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.308+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.308+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.308+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.309+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.309+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.309+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.309+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.310+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.310+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.310+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.310+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.311+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.311+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.311+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.311+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.312+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.312+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.312+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.312+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.313+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.313+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.313+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.313+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.314+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.314+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.314+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.314+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.315+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.315+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.315+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.315+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.316+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.316+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.316+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.316+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.317+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.317+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.317+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.317+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.318+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.318+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.318+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.318+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.319+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.319+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.308+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.308+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.311+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.311+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.311+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.311+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.312+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.312+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.312+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.312+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.313+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.313+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.314+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.314+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.314+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.314+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.315+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.315+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.315+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.315+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.315+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.315+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.316+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.316+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.317+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.317+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.317+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.317+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.317+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.317+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.318+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.318+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.302+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.302+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.302+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.302+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.303+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.303+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.305+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.305+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.305+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.305+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.306+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.306+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.306+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.306+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.307+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.307+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.307+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.307+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.308+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.308+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.308+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.308+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.309+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.309+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.309+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.309+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.310+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.310+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.310+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.310+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.311+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.311+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.311+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.311+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.312+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.312+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.312+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.312+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.313+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.313+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.313+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.313+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.314+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.314+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.314+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.314+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.315+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.315+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.315+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.315+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.316+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.316+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.316+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.316+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.317+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.317+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.317+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.317+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.318+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.318+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.318+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.318+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.319+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.319+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.319+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.319+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.319+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.319+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.320+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.320+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.320+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.320+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.320+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.320+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.321+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.321+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.321+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.321+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.322+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.322+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.322+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.322+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.323+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.323+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.323+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.323+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.324+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.324+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.324+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.324+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.325+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.325+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.325+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.325+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.326+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.326+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.326+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.326+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.327+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.327+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.327+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.327+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.328+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.328+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.328+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.328+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.329+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.329+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.329+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.329+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.330+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.330+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.331+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.331+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.331+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.331+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.332+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.332+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.332+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.332+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.333+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.333+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.333+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.333+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.334+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.334+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.334+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.334+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.335+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.335+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.335+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.335+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.336+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.336+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.336+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.336+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.337+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.337+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.337+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.337+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.338+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.338+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.338+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.338+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.339+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.339+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.339+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.339+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.340+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.340+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.340+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.340+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.341+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.341+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.341+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.341+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.342+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.342+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.342+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.342+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.343+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.343+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.343+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.343+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.344+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.344+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.345+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.345+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.345+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.345+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.346+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.346+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.346+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.346+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.347+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.347+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.347+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.347+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.348+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.348+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.348+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.348+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.349+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.349+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.349+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.349+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.350+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.350+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.350+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.350+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.351+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.351+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.351+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.351+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.335+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.335+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.335+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.335+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.337+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.337+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.320+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.320+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.321+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.321+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.321+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.321+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.322+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.322+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.322+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.322+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.323+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.323+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.323+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.323+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.324+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.324+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.324+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.324+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.325+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.325+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.325+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.325+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.326+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.326+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.326+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.326+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.327+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.327+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.327+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.327+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.328+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.328+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.328+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.328+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.329+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.329+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.329+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.329+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.330+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.330+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.330+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.330+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.331+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.331+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.331+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.331+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.332+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.332+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.332+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.332+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.332+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.332+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.333+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.333+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.335+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.335+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.335+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.335+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.336+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.336+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.336+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.336+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.337+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.337+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.337+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.337+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.337+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.337+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.338+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.338+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.338+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.338+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.339+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.339+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.339+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.339+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.340+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.340+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.340+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.340+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.341+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.341+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.341+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.341+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.342+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.342+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.342+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.342+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.343+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.343+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.343+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.343+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.344+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.344+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.344+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.344+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.345+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.345+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.345+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.345+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.346+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.346+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.346+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.346+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.347+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.347+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.347+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.347+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.348+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.348+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.348+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.348+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.349+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.349+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.349+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.349+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.350+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.350+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.350+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.350+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.351+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.351+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.351+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.351+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.352+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.352+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.352+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.352+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.352+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.352+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.352+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.352+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.353+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.353+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.353+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.353+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.353+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.353+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.353+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.353+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.354+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.354+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.354+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.354+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.354+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.354+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.354+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.354+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.355+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.355+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.355+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.355+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.355+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.355+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.355+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.355+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.356+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.356+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.356+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.356+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.356+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.356+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.357+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.357+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.357+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.357+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.357+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.357+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.357+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.357+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.357+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.357+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.358+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.358+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.358+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.358+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.358+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.358+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.358+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.358+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.359+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.359+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.359+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.359+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.359+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.359+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.359+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.359+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.360+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.360+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.360+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.360+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.361+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.361+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.361+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.361+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.361+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.361+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.361+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.361+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.362+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.362+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.362+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.362+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.362+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.362+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.362+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.362+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.363+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.363+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.363+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.363+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.363+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.363+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.363+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.363+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.364+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.364+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.364+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.364+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.364+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.364+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.364+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.364+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.365+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.365+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.365+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.365+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.365+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.365+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.365+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.365+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.366+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.366+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.366+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.366+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.366+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.366+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.366+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.366+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.367+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.367+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.367+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.367+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.367+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.367+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.367+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.367+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.367+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.368+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.368+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.368+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.368+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.368+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.368+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.368+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.368+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.368+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.369+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.369+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.369+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.369+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.369+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.369+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.369+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.369+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.370+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.370+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.370+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.370+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.371+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.371+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.371+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.371+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.371+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.371+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.372+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.372+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.372+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.372+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.372+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.372+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.372+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.372+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.373+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.373+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.373+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.373+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.373+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.373+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.373+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.373+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.374+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.374+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.374+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.374+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.374+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.374+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.375+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.375+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.375+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.375+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.375+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.375+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.375+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.375+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.376+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.376+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.376+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.376+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.376+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.376+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.376+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.376+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.377+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.377+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.377+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.377+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.377+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.377+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.377+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.377+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.378+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.378+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.379+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.379+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.379+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.379+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.380+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.380+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.380+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.380+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.381+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.381+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.381+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.381+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.382+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.382+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.382+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.382+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.383+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.383+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.383+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.383+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.384+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.384+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.384+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.384+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.385+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.385+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.385+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.385+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.386+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.386+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.386+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.386+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.387+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.387+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.387+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.387+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.388+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.388+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.388+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.388+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.389+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.389+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.389+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.389+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.390+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.390+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.390+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.390+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.391+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.391+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.391+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.391+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.391+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.391+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.392+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.392+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.393+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.393+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.393+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.393+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.394+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.394+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.396+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.396+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.396+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.396+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.397+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.397+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.397+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.397+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.397+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.397+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.398+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.398+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.398+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.398+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.399+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.399+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.399+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.400+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.400+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.400+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.400+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.401+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.401+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.401+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.402+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.402+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.402+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.402+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.403+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.403+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.403+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.403+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.404+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.404+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.404+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.404+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.405+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.405+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.405+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.405+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.405+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.405+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.406+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.406+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.406+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.406+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.407+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.407+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.407+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.407+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.408+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.408+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.409+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.409+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.409+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.409+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.410+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.410+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.410+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.410+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.410+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.410+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.411+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.411+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.411+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.411+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.412+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.412+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.412+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.412+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.413+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.413+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.413+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.413+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.414+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.414+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.414+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.414+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.415+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.415+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.415+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.415+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.416+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.416+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.416+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.416+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.417+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.417+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.417+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.417+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.418+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.418+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.418+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.418+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.419+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.419+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.419+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.419+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.420+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.420+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.420+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.420+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.420+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.420+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.421+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.421+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.421+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.421+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.422+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.422+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.422+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.422+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.423+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.423+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.423+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.423+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.424+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.424+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.424+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.424+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.425+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.386+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.386+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.389+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.389+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.411+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.411+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.411+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.411+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.411+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.411+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.412+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.412+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.412+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.412+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.413+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.413+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.413+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.413+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.415+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.415+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.418+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.418+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.419+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.419+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.420+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.420+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.423+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.423+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.423+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.423+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.377+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.377+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.378+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.378+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.378+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.378+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.379+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.379+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.379+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.379+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.380+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.380+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.380+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.380+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.381+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.381+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.381+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.381+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.382+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.382+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.382+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.382+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.383+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.383+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.383+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.383+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.384+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.384+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.384+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.384+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.384+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.384+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.385+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.385+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.385+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.385+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.386+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.386+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.386+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.386+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.387+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.387+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.387+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.387+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.388+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.388+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.388+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.388+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.389+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.389+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.389+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.389+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.390+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.390+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.391+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.391+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.391+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.391+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.392+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.392+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.392+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.392+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.393+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.393+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.393+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.393+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.394+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.394+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.394+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.394+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.395+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.395+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.395+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.395+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.396+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.396+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.396+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.396+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.397+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.397+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.397+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.397+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.398+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.398+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.398+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.398+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.399+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.399+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.399+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.399+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.400+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.400+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.400+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.400+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.401+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.401+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.401+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.401+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.403+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.403+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.403+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.403+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.404+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.404+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.404+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.404+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.405+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.405+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.405+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.405+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.406+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.406+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.406+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.406+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.407+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.407+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.407+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.407+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.407+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.407+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.408+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.408+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.409+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.409+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.409+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.409+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.410+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.410+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.410+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.410+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.411+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.411+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.411+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.411+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.413+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.413+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.413+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.413+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.414+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.414+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.414+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.414+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.415+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.415+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.415+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.415+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.416+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.416+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.416+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.416+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.417+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.417+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.418+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.418+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.418+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.418+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.419+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.419+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.419+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.419+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.420+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.420+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.420+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.420+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.421+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.421+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.422+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.422+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.422+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.422+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.422+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.422+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.425+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.426+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.426+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.426+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.426+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.427+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.427+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.427+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.427+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.428+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.428+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.428+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.428+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.429+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.429+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.429+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.429+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.430+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.430+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.430+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.430+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.431+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.431+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.431+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.431+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.423+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.423+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.423+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.423+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.424+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.424+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.424+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.424+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.425+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.425+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.425+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.425+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.426+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.426+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.426+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.426+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.427+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.427+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.427+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.427+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.428+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.428+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.428+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.428+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.429+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.429+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.429+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.429+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.430+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.430+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.430+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.430+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.431+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.431+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.432+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.432+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.432+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.432+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.432+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.432+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.432+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.432+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.433+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.433+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.433+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.433+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.433+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.433+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.433+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.433+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.434+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.434+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.434+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.434+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.434+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.434+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.434+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.434+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.434+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.434+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.434+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.434+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.435+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.435+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.435+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.435+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.435+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.435+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.435+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.435+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.436+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.436+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.436+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.436+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.436+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.436+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.436+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.436+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.436+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.436+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.436+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.436+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.437+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.437+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.437+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.437+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.437+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.437+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.437+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.437+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.437+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.437+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.437+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.438+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.438+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.438+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.438+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.438+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.438+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.438+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.438+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.438+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.439+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.439+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.439+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.439+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.440+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.440+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.440+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.440+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.441+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.441+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.441+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.441+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.442+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.442+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.442+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.442+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.443+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.443+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.443+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.443+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.444+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.444+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.444+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.444+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.445+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.445+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.445+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.445+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.446+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.446+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.446+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.446+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.447+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.447+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.447+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.447+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.448+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.448+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.448+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.448+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.449+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.449+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.449+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.449+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.450+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.450+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.450+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.450+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.451+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.451+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.451+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.451+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.452+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.452+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.452+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.452+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.453+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.453+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.453+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.453+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.454+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.454+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.454+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.454+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.455+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.455+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.455+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.455+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.456+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.456+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.456+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.456+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.457+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.457+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.457+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.457+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.458+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.458+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.458+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.458+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.459+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.459+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.459+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.459+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.460+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.460+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.461+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.461+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.461+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.461+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.462+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.462+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.462+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.462+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.463+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.463+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.463+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.463+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.464+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.464+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.464+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.464+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.465+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.465+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.465+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.465+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.466+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.466+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.466+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.466+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.467+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.467+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.467+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.467+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.468+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.468+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.468+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.468+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.439+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.439+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.445+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.445+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.447+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.447+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.451+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.451+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.462+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.462+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.438+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.438+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.440+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.440+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.442+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.442+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.442+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.442+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.443+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.443+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.444+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.444+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.444+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.444+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.445+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.445+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.445+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.445+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.446+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.446+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.446+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.446+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.447+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.447+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.447+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.447+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.448+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.448+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.448+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.448+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.449+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.449+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.449+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.449+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.450+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.450+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.450+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.450+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.451+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.451+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.451+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.451+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.452+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.452+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.452+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.452+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.453+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.453+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.453+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.453+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.454+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.454+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.454+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.454+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.456+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.456+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.456+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.456+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.457+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.457+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.457+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.457+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.458+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.458+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.458+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.458+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.459+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.459+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.459+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.459+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.460+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.460+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.460+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.460+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.461+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.461+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.461+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.461+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.462+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.462+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.462+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.462+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.463+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.463+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.463+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.463+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.464+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.464+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.464+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.464+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.465+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.465+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.465+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.465+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.466+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.466+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.467+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.467+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.467+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.467+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.468+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.468+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.468+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.468+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.469+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.469+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.469+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.470+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.470+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.470+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.470+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.470+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.470+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.470+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.471+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.471+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.471+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.471+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.471+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.471+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.471+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.471+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.472+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.472+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.472+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.472+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.472+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.472+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.472+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.472+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.473+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.473+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.473+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.473+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.473+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.473+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.473+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.473+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.474+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.474+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.474+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.474+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.474+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.474+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.474+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.474+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.475+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.475+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.475+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.475+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.475+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.475+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.475+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.475+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.476+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.476+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.476+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.476+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.476+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.476+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.476+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.476+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.477+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.477+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.477+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.477+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.477+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.477+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.477+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.477+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.478+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.478+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.478+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.478+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.478+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.478+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.478+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.478+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.479+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.479+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.479+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.479+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.479+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.479+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.479+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.479+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.479+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.479+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.480+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.480+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.480+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.480+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.480+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.480+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.480+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.480+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.481+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.481+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.481+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.481+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.481+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.481+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.481+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.481+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.482+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.482+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.482+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.482+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.482+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.482+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.483+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.483+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.483+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.483+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.483+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.483+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.483+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.483+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.484+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.484+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.484+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.484+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.484+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.484+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.484+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.484+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.485+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.485+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.485+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.485+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.485+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.485+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.485+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.485+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.486+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.486+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.486+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.486+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.487+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.487+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.487+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.487+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.487+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.487+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.487+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.487+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.488+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.488+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.488+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.488+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.488+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.488+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.488+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.488+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.489+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.489+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.489+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.489+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.489+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.489+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.489+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.489+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.490+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.490+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.490+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.490+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.490+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.490+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.490+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.490+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.491+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.491+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.491+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.491+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.491+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.491+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.491+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.491+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.492+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.492+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.492+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.492+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.492+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.492+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.493+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.493+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.493+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.493+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.493+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.493+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.493+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.493+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.494+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.494+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.494+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.494+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.494+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.494+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.494+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.494+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.495+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.495+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.495+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.495+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.495+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.495+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.495+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.495+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.496+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.496+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.496+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.496+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.496+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.496+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.497+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.497+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.497+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.497+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.498+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.498+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.498+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.498+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.499+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.499+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.499+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.499+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.499+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.500+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.500+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.500+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.501+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.501+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.501+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.501+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.502+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.502+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.502+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.502+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.503+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.503+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.503+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.503+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.504+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.504+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.504+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.504+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.505+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.505+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.505+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.505+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.506+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.506+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.506+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.506+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.507+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.507+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.507+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.507+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.508+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.508+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.508+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.508+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.509+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.509+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.509+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.509+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.510+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.510+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.510+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.510+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.511+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.511+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.511+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.511+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.512+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.512+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.512+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.512+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.513+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.513+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.513+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.513+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.514+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.514+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.514+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.514+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.515+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.515+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.515+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.515+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.516+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.516+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.516+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.516+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.517+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.517+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.518+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.518+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.518+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.518+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.519+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.519+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.519+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.519+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:01.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.496+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.496+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.497+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.497+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.497+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.497+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.498+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.498+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.498+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.498+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.499+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.499+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.499+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.499+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.500+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.500+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.500+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.500+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.501+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.501+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.501+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.501+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.502+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.502+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.502+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.502+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.503+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.503+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.503+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.503+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.504+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.504+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.504+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.504+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.505+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.505+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.505+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.505+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.506+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.506+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.506+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.506+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.507+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.507+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.507+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.507+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.508+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.508+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.508+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.508+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.509+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.509+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.509+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.509+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.510+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.510+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.510+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.510+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.511+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.511+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.511+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.511+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.512+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.512+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.512+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.512+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.512+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.512+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.513+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.513+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.514+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.514+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.514+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.514+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.515+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.515+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.515+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.515+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.516+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.516+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.516+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.516+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.517+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.517+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.517+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.517+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.518+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.518+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.518+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.518+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.519+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.519+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.519+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.519+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.520+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.520+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.520+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.520+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.520+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.520+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.520+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.520+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.521+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.521+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.521+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.521+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.521+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.521+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.521+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.521+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.521+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.521+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.522+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.522+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.522+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.522+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.522+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.522+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.522+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.523+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.523+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.523+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.523+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.523+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.523+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.523+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.523+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.523+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.523+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.523+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.524+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.524+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.524+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.524+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.524+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.524+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.524+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.524+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.524+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.524+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.525+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.525+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.525+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.525+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.525+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.525+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.525+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.525+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.526+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.526+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.526+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.526+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.526+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.526+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.526+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.526+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.527+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.527+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.527+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.527+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.527+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.527+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.527+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.527+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.528+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.528+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.528+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.528+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.528+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.528+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.528+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.528+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.529+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.529+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.529+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.529+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.529+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.529+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.529+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.529+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.530+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.530+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.530+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.530+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.530+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.530+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.530+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.530+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.531+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.531+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.531+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.531+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.531+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.531+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.531+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.531+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.532+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.532+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.532+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.532+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.532+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.532+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.532+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.532+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.533+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.533+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.533+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.533+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.533+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.533+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.533+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.533+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.534+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.534+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.534+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.534+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.534+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.534+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.534+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.534+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.535+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.535+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.535+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.535+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.535+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.535+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.535+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.536+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.536+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.536+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.536+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.536+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.536+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.536+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.536+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.536+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.537+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.537+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.537+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.537+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.538+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.538+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.538+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.538+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.538+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.538+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.538+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.538+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.538+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.538+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.539+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.539+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.539+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.539+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.539+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.539+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.540+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.540+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.540+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.540+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.540+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.540+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.540+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.540+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.540+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.540+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.541+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.541+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.541+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.541+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.541+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.541+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.541+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.541+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.542+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.542+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.543+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.543+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.543+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.543+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.543+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.543+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.544+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.544+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.544+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.544+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.544+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.544+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.544+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.544+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.545+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.545+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.545+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.545+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.545+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.545+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.545+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.545+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.546+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.546+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.546+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.546+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.546+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.546+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.546+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.546+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.547+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.547+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.547+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.547+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.547+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.547+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.547+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.547+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.548+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.548+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.548+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.548+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.548+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.548+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.548+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.548+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.549+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.549+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.549+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.549+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.549+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.549+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.549+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.549+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.550+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.550+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.550+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.550+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.550+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.550+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.550+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.550+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.551+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.551+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.551+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.551+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.551+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.551+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.551+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.551+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.552+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.552+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.552+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.552+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.553+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.553+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.553+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.553+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.554+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.554+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.554+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.554+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.555+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.555+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.555+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.555+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.556+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.556+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.556+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.556+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.557+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.557+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.557+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.557+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.552+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.552+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.553+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.553+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.553+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.553+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.553+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.553+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.554+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.554+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.554+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.555+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.555+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.555+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.556+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.556+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.556+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.556+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.557+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.557+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.557+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.557+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.558+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.558+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.558+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.558+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.558+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.558+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.558+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.558+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.559+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.559+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.559+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.559+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.559+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.559+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.559+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.559+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.560+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.560+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.560+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.560+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.560+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.560+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.560+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.560+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.561+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.561+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.561+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.561+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.562+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.562+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.562+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.562+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.562+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.562+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.563+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.563+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.563+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.563+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.563+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.563+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.563+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.563+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.564+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.564+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.564+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.564+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.564+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.564+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.564+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.564+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.565+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.565+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.565+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.565+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.565+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.565+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.565+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.565+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.566+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.566+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.566+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.566+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.566+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.566+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.566+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.566+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.567+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.567+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.567+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.567+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.567+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.567+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.567+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.567+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.567+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.567+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.568+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.568+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.568+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.568+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.568+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.568+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.568+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.568+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.569+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.569+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.569+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.569+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.569+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.569+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.569+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.569+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.570+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.570+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.570+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.570+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.570+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.570+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.570+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.570+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.571+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.571+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.571+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.571+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.571+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.571+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.571+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.571+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.572+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.572+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.572+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.572+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.572+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.572+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.572+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.572+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.572+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.572+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.572+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.572+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.573+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.573+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.573+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.573+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.573+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.573+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.573+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.573+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.574+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.574+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.574+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.574+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.574+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.574+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.574+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.574+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.575+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.575+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.575+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.575+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.575+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.575+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.575+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.575+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.576+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.576+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.576+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.576+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.576+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.576+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.576+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.576+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.577+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.577+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.577+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.577+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.577+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.577+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.577+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.577+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.578+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.578+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.578+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.578+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.578+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.578+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.578+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.578+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.579+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.579+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.579+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.579+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.579+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.579+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.579+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.579+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.580+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.580+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.580+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.580+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.580+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.580+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.580+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.580+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.581+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.581+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.581+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.581+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.581+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.581+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.581+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.581+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.582+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.582+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.582+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.582+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.582+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.582+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.582+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.582+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.583+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.583+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.583+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.583+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.583+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.583+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.583+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.583+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.584+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.584+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.584+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.584+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.584+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.584+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.584+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.584+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.585+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.585+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.585+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.585+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.585+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.585+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.585+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.585+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.586+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.586+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.586+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.586+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.586+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.586+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.586+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.586+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.587+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.587+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.587+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.587+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.587+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.587+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.587+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.587+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.587+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.587+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.587+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.587+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.588+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.588+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.588+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.588+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.588+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.588+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.588+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.588+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.589+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.589+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.589+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.589+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.589+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.589+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.589+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.589+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.590+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.590+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.590+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.590+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.590+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.590+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.590+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.590+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.591+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.591+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.591+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.591+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.591+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.591+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.591+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.591+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.592+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.592+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.592+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.592+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.592+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.592+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.592+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.592+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.593+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.593+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.593+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.593+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.593+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.593+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.593+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.593+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.594+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.594+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.594+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.594+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.594+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.594+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.594+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.594+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.595+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.595+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.595+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.595+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.595+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.595+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.595+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.595+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.596+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.596+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.596+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.596+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.596+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.596+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.597+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.597+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.597+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.597+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.597+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.597+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.597+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.597+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.598+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.598+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.598+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.598+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.598+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.598+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.598+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.598+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.599+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.599+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.599+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.599+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.599+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.599+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.599+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.599+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.600+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.600+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.600+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.600+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.600+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.600+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.600+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.600+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.601+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.601+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.602+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.602+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.602+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.602+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.602+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.602+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.603+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.603+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.603+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.603+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.604+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.604+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.604+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.604+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.604+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.604+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.604+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.604+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.605+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.605+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.605+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.605+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.605+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.605+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.606+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.606+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.606+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.606+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.607+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.607+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.607+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.607+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.608+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.608+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.608+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.608+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.609+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.609+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.609+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.609+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.610+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.610+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.610+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.610+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.611+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.611+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.608+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.608+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.611+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.611+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.605+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.605+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.605+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.605+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.606+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.606+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.606+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.606+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.607+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.607+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.607+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.607+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.608+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.608+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.608+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.608+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.609+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.609+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.609+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.609+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.610+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.610+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.610+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.610+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.611+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.611+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.611+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.611+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.611+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.611+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.611+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.612+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.612+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.612+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.612+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.612+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.612+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.612+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.612+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.613+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.613+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.613+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.613+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.613+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.613+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.613+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.613+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.613+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.613+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.614+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.614+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.614+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.614+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.614+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.614+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.614+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.614+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.615+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.615+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.615+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.615+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.615+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.615+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.615+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.615+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.616+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.616+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.616+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.616+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.616+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.616+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.616+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.616+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.617+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.617+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.617+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.617+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.617+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.617+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.617+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.617+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.618+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.618+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.618+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.618+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.619+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.619+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.619+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.619+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.619+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.619+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.620+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.620+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.620+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.620+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.620+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.620+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.620+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.620+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.621+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.621+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.621+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.621+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.621+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.621+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.621+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.621+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.622+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.622+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.622+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.622+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.622+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.622+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.622+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.622+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.623+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.623+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.623+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.623+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.623+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.623+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.623+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.623+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.624+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.624+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.624+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.624+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.624+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.624+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.624+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.624+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.625+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.625+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.625+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.625+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.625+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.625+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.625+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.625+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.625+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.625+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.626+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.626+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.626+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.626+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.626+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.626+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.626+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.626+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.626+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.626+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.627+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.627+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.627+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.627+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.627+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.627+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.627+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.627+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.627+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.627+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.628+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.628+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.628+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.628+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.628+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.628+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.628+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.628+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.628+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.628+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.629+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.629+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.629+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.629+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.629+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.629+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.629+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.629+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.630+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.630+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.630+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.630+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.630+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.630+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.630+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.630+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.630+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.630+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.630+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.630+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.631+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.631+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.631+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.631+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.631+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.631+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.631+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.631+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.632+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.632+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.632+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.632+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.633+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.633+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.633+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.633+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.633+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.633+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.633+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.633+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.634+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.634+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.634+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.634+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.634+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.634+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.634+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.634+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.635+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.635+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.635+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.635+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.635+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.635+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.635+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.635+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.636+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.636+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.636+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.636+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.636+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.636+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.636+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.636+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.636+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.636+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.637+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.637+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.637+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.637+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.637+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.637+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.637+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.637+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.638+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.638+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.638+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.638+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.638+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.638+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.638+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.638+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.638+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.638+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.639+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.639+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.639+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.639+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.639+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.639+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.639+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.639+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.640+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.640+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.640+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.640+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.640+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.640+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.640+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.640+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.640+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.640+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.641+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.641+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.641+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.641+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.641+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.641+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.641+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.641+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.642+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.642+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.642+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.642+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.642+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.642+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.642+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.642+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.643+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.643+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.643+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.643+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.643+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.643+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.643+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.643+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.644+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.644+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.644+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.644+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.644+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.644+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.644+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.644+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.644+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.644+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.645+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.645+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.645+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.645+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.645+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.645+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.645+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.645+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.646+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.646+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.646+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.646+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.646+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.646+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.646+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.646+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.647+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.647+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.647+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.647+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.647+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.647+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.647+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.647+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.648+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.648+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.648+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.648+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.648+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.648+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.648+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.648+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.648+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.648+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.649+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.649+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.649+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.649+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.649+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.649+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.649+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.649+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.650+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.650+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.650+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.650+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.650+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.650+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.650+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.650+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.651+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.651+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.651+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.651+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.651+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.651+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.651+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.651+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.652+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.652+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.652+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.652+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.653+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.653+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.653+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.654+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.654+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.654+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.654+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.654+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.654+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.654+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.654+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.655+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.655+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.655+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.655+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.655+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.655+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.655+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.655+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.656+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.656+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.656+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.656+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.656+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.656+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.656+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.656+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.657+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.657+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.657+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.657+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.657+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.657+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.657+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.657+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.658+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.658+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.658+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.658+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.658+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.658+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.658+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.658+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.659+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.659+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.659+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.659+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.659+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.659+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.659+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.659+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.659+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.659+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.660+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.660+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.660+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.660+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.660+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.660+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.660+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.660+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.660+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.660+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.661+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.661+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.661+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.661+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.661+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.661+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.662+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.662+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.662+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.662+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.662+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.662+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.662+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.662+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.663+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.663+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.663+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.663+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.663+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.663+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.663+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.663+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.664+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.664+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.664+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.664+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.664+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.664+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.665+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.665+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.665+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.665+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.665+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.665+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.665+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.665+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.666+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.666+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.666+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.666+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.666+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.666+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.666+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.666+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.667+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.667+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.667+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.667+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.667+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.667+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.668+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.668+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.669+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.669+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.669+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.669+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.669+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.669+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.669+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.669+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.670+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.670+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.670+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.670+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.670+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.670+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.670+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.670+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.671+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.671+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.671+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.671+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.671+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.671+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.671+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.671+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.672+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.672+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.672+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.672+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.672+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.672+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.672+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.672+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.673+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.673+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.673+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.673+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.673+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.673+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.673+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.673+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.674+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.674+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.674+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.674+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.674+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.674+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.674+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.674+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.675+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.675+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.675+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.675+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.676+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.676+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.676+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.676+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.676+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.676+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.676+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.676+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.677+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.677+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.677+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.677+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.677+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.677+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.677+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.677+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.678+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.678+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.678+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.678+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.678+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.678+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.679+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.679+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.679+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.679+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.679+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.679+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.680+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.680+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.680+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.680+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.681+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.681+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.681+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.681+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.681+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.681+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.682+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.682+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.682+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.682+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.682+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.682+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.683+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.683+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.683+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.683+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.683+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.683+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.683+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.683+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.684+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.684+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.684+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.684+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.685+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.685+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.685+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.685+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.685+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.685+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.685+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.685+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.686+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.686+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.686+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.686+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.686+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.686+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.686+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.686+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.686+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.686+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.687+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.687+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.687+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.687+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.687+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.687+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.688+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.688+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.688+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.688+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.688+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.688+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.688+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.688+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.688+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.688+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.689+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.689+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.689+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.689+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.689+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.689+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.690+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.690+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.690+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.690+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.690+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.690+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.690+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.690+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.691+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.691+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.691+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.691+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.691+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.691+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.691+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.692+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.692+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.692+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.692+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.692+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.692+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.692+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.692+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.693+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.693+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.693+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.693+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.693+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.693+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.693+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.693+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.693+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.694+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.694+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.694+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.694+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.694+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.694+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.694+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.694+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.695+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.695+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.695+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.695+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.695+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.695+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.695+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.696+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.696+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.696+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.696+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.696+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.696+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.697+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.697+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.697+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.697+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.698+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.698+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.698+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.698+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.698+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.698+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.698+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.698+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.698+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.698+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.698+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.698+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.699+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.699+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.699+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.699+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.699+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.699+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.699+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.699+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.700+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.700+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.700+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.700+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.700+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.700+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.700+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.700+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.701+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.701+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.701+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.701+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.701+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.701+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.701+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.701+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.702+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.702+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.702+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.702+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.702+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.702+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.702+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.702+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.703+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.703+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.703+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.703+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.703+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.703+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.704+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.704+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.704+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.704+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.705+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.705+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.705+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.705+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.705+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.705+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.705+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.705+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.706+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.706+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.706+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.706+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.706+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.706+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.706+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.706+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.706+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.706+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.707+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.707+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.707+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.707+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.707+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.707+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.707+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.707+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.708+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.708+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.708+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.708+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.708+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.708+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.708+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.708+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.708+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.708+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.708+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.708+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.709+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.709+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.709+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.709+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.709+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.709+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.709+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.709+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.709+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.709+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.710+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.710+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.710+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.710+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.710+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.710+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.710+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.710+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.711+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.711+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.711+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.711+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.711+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.711+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.711+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.711+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.712+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.712+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.712+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.712+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.712+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.712+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.712+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.713+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.713+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.713+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.713+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.713+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.713+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.713+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.714+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.714+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.714+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.714+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.714+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.714+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.714+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.714+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.715+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.715+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.715+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.715+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.715+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.715+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.716+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.716+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.716+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.716+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.716+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.716+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.717+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.717+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.717+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.717+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.717+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.717+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.717+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.717+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.718+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.718+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.718+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.718+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.718+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.718+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.719+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.719+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.719+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.719+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.719+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.719+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.719+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.719+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.720+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.720+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.720+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.720+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.720+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.720+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.720+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.720+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.721+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.721+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.721+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.721+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.721+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.721+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.721+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.721+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.721+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.721+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.721+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.721+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.722+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.722+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.722+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.722+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.722+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.722+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.722+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.722+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.723+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.723+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.723+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.723+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.723+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.723+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.723+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.723+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.724+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.724+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.724+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.724+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.724+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.724+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.724+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.724+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.725+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.725+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.725+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.725+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.725+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.725+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.725+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.725+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.725+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.725+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.726+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.726+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.726+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.726+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.726+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.726+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.726+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.727+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.727+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.727+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.727+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.727+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.727+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.727+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.727+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.727+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.728+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.728+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.728+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.728+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.728+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.728+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.729+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.729+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.729+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.729+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.729+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.729+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.730+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.730+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.730+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.730+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.730+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.730+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.731+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.731+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.732+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.732+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.732+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.732+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.732+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.732+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.732+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.732+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.733+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.733+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.733+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.733+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.733+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.733+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.733+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.733+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.734+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.734+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.734+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.734+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.734+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.734+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.734+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.734+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.735+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.735+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.735+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.735+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.735+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.735+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.735+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.735+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.736+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.736+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.736+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.736+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.736+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.736+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.736+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.737+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.737+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.737+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.737+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.737+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.737+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.737+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.737+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.738+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.738+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.738+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.738+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.738+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.738+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.738+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.738+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.738+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.738+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.739+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.739+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.739+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.739+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.739+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.739+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.739+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.739+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.740+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.740+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.740+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.740+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.740+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.740+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.740+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.740+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.741+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.741+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.741+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.741+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.741+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.741+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.741+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.741+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.742+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.742+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.742+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.742+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.742+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.742+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.742+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.742+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.742+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.742+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.743+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.743+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.743+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.743+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.743+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.743+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.743+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.743+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.744+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.744+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.744+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.744+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.745+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.745+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.745+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.745+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.746+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.746+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.746+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.746+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.746+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.746+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.747+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.747+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.747+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.747+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.748+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.748+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.749+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.749+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.749+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.749+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.750+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.750+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.750+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.750+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.751+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.751+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.751+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.751+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.752+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.752+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.752+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.752+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.753+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.753+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.753+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.753+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.754+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.754+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.754+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.754+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.755+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.755+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.755+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.755+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.756+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.756+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.756+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.756+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.757+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.757+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.757+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.757+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.758+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.758+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.758+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.758+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.759+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.759+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.759+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.759+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.760+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.760+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.760+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.760+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.761+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.761+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.761+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.761+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.762+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.762+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.762+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.762+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.763+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.763+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.763+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.763+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.764+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.764+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.764+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.764+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.765+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.765+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.765+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.765+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.766+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.766+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.766+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.766+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.767+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.767+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.767+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.767+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.768+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.768+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.744+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.744+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.744+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.744+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.745+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.745+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.745+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.745+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.746+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.746+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.746+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.746+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.747+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.747+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.747+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.747+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.748+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.748+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.748+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.748+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.749+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.749+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.749+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.749+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.750+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.750+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.750+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.750+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.751+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.751+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.751+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.751+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.753+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.753+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.753+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.753+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.754+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.754+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.754+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.754+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.755+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.755+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.755+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.755+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.756+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.756+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.756+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.756+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.758+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.758+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.758+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.758+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.759+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.759+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.759+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.759+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.760+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.760+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.760+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.760+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.761+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.761+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.761+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.761+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.762+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.762+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.762+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.762+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.763+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.763+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.763+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.763+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.764+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.764+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.764+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.764+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.765+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.765+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.765+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.765+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.766+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.766+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.766+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.766+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.767+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.767+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.767+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.767+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.768+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.768+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.768+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.768+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.769+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.769+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.769+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.769+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.769+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.769+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.769+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.769+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.769+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.769+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.770+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.770+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.770+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.770+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.770+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.770+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.770+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.770+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.771+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.771+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.771+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.771+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.771+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.771+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.772+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.772+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.772+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.772+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.772+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.772+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.772+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.772+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.773+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.773+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.773+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.773+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.773+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.773+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.773+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.773+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.774+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.774+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.774+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.774+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.774+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.774+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.774+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.774+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.775+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.775+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.775+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.775+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.775+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.775+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.776+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.776+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.776+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.776+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.776+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.776+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.776+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.776+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.777+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.777+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.777+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.777+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.777+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.777+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.777+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.777+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.778+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.778+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.778+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.778+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.778+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.778+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.778+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.778+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.779+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.779+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.779+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.779+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.779+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.779+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.780+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.780+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.780+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.780+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.780+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.780+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.780+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.780+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.780+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.781+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.781+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.781+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.781+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.782+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.782+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.782+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.782+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.782+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.782+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.782+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.782+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.782+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.782+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.782+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.782+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.783+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.783+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.783+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.783+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.783+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.783+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.783+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.783+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.784+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.784+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.784+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.784+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.784+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.784+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.785+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.785+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.785+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.785+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.785+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.785+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.785+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.785+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.785+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.785+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.786+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.786+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.786+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.786+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.786+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.786+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.786+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.786+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.786+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.786+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.786+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.786+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.787+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.787+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.787+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.787+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.787+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.787+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.788+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.788+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.788+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.788+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.788+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.788+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.788+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.788+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.789+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.789+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.789+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.789+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.789+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.789+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.789+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.789+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.790+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.790+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.790+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.790+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.790+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.790+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.790+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.790+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.791+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.791+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.791+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.791+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.791+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.791+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.791+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.791+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.793+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.793+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.793+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.793+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.793+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.793+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.793+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.793+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.794+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.794+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.794+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.794+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.794+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.794+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.794+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.794+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.795+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.795+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.795+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.795+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.796+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.796+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.796+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.796+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.796+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.796+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.796+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.796+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.796+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.796+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.797+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.797+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.797+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.797+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.797+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.797+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.797+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.797+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.798+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.798+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.798+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.798+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.799+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.799+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.799+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.799+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.799+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.799+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.799+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.799+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.800+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.800+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.801+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.801+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.801+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.801+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.802+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.802+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.802+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.802+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.803+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.803+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.803+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.803+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.804+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.804+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.804+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.804+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.805+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.805+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.805+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.805+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.806+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.806+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.806+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.806+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.808+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.808+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.808+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.808+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.809+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.809+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.809+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.809+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.810+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.810+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.810+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.810+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.811+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.811+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.811+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.811+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.812+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.812+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.812+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.812+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.813+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.813+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.813+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.813+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.814+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.814+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.814+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.814+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.814+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.814+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.815+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.815+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.815+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.815+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.816+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.816+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.816+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.816+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.817+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.817+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.817+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.817+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.818+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.818+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.818+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.818+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.819+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.819+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.819+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.819+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.820+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.820+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.820+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.820+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.821+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.821+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.821+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.821+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.821+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.821+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.822+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.822+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.822+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.822+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.823+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.823+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.823+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.823+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.824+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.824+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.824+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.824+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.825+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.825+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.825+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.825+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.826+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.826+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.826+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.826+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.827+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.827+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.827+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.827+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.828+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.828+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.828+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.828+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.829+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.829+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.829+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.829+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.830+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.830+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.830+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.830+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.832+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.832+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.832+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.832+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.833+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.833+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.833+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.833+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.834+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.834+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.834+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.834+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.835+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.835+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.835+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.835+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.836+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.836+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.836+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.836+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.837+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.837+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.837+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.837+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.838+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.838+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.838+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.838+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.839+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.839+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.839+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.839+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.840+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.840+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.840+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.840+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.842+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.842+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.842+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.842+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.843+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.843+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.843+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.843+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.844+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.844+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.844+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.844+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.801+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.801+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.801+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.801+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.806+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.806+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.806+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.806+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.808+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.808+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.812+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.812+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.812+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.812+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.825+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.825+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.826+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.826+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.826+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.826+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.832+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.832+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.834+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.834+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.834+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.834+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.841+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.841+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.842+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.842+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.842+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.842+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.843+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.843+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.843+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.843+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.844+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.844+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.844+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.844+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.800+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.800+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.801+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.801+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.801+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.801+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.802+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.802+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.802+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.802+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.803+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.803+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.803+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.803+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.804+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.804+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.804+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.804+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.805+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.805+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.805+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.805+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.806+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.806+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.806+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.806+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.808+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.808+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.808+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.808+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.809+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.809+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.809+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.809+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.811+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.811+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.811+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.811+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.812+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.812+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.812+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.812+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.813+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.813+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.813+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.813+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.814+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.814+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.814+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.814+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.815+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.815+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.815+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.815+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.816+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.816+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.816+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.816+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.817+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.817+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.817+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.817+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.819+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.819+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.819+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.819+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.820+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.820+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.820+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.820+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.821+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.821+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.821+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.821+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.822+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.822+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.822+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.822+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.822+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.822+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.823+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.823+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.823+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.823+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.824+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.824+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.824+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.824+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.825+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.825+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.825+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.825+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.826+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.826+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.826+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.826+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.827+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.827+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.827+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.827+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.828+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.828+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.828+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.828+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.829+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.829+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.829+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.829+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.830+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.830+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.830+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.830+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.831+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.831+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.831+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.831+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.832+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.832+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.832+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.832+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.833+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.833+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.833+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.833+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.834+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.834+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.834+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.834+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.835+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.835+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.836+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.836+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.836+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.836+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.837+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.837+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.838+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.838+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.838+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.838+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.839+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.839+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.839+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.839+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.840+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.840+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.840+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.840+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.841+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.841+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.841+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.841+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.842+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.842+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.842+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.842+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.843+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.843+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.843+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.843+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.844+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.844+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.844+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.844+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.845+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.845+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.845+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.845+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.846+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.846+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.846+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.846+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.848+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.848+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.846+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.846+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.846+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.846+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.847+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.847+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.847+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.847+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.848+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.848+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.848+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.848+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.850+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.850+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.851+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.851+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.851+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.851+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.852+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.852+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.852+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.852+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.853+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.853+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.853+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.853+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.854+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.854+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.854+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.854+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.856+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.856+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.856+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.856+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.857+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.857+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.857+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.857+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.858+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.858+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.858+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.858+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.859+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.859+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.859+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.859+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.860+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.860+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.860+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.860+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.861+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.861+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.861+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.861+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.862+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.862+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.862+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.862+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.863+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.863+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.864+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.864+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.864+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.864+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.865+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.865+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.865+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.865+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.866+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.866+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.866+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.866+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.867+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.867+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.867+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.867+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.868+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.868+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.868+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.868+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.869+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.869+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.869+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.869+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.870+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.870+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.870+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.870+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.871+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.871+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.871+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.871+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.872+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.872+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.872+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.872+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.873+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.873+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.873+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.873+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.874+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.874+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.874+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.874+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.875+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.875+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.875+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.875+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.875+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.875+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.876+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.876+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.876+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.876+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.877+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.877+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.878+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.878+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.878+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.878+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.879+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.879+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.879+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.879+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.880+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.880+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.880+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.880+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.881+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.881+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.881+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.881+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.882+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.882+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.882+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.882+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.883+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.883+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.883+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.883+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.884+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.884+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.854+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.854+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.854+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.854+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.855+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.855+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.856+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.856+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.856+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.856+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.866+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.866+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.866+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.866+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.870+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.870+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.870+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.870+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.849+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.849+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.849+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.849+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.850+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.850+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.850+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.851+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.851+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.851+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.851+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.851+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.852+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.852+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.852+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.853+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.853+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.853+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.854+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.854+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.854+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.854+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.855+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.855+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.855+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.855+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.856+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.856+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.856+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.856+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.857+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.857+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.857+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.857+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.858+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.858+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.858+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.858+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.859+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.859+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.859+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.859+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.859+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.860+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.860+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.861+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.861+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.861+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.861+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.862+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.862+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.862+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.862+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.862+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.863+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.863+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.863+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.863+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.863+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.864+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.864+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.865+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.865+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.865+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.865+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.866+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.866+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.867+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.867+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.867+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.867+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.868+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.868+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.868+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.868+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.869+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.869+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.869+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.869+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.870+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.870+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.870+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.870+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.871+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.871+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.871+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.871+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.872+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.872+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.872+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.872+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.873+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.873+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.873+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.873+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.874+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.874+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.874+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.874+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.875+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.875+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.875+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.875+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.876+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.876+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.876+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.876+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.877+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.877+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.877+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.877+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.878+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.878+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.878+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.878+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.879+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.879+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.879+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.879+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.880+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.880+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.880+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.880+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.881+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.881+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.881+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.881+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.883+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.883+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.884+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.884+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.885+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.885+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.885+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.885+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.886+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.886+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.886+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.886+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.887+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.887+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.887+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.887+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.888+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.888+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.888+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.888+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.888+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.888+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.889+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.889+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.890+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.890+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.890+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.890+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.891+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.891+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.891+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.891+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.892+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.892+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.892+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.892+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.893+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.893+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.893+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.893+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.894+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.894+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.894+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.894+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.895+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.895+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.895+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.895+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.896+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.896+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.896+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.896+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.897+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.897+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.897+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.897+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.898+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.898+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.898+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.898+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.899+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.899+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.899+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.899+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.900+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.900+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.900+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.900+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.901+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.901+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.901+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.901+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.902+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.902+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.902+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.902+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.903+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.903+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.903+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.903+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.904+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.904+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.904+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.904+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.905+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.905+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.905+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.905+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.906+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.906+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.906+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.906+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.907+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.907+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.907+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.907+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.908+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.908+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.908+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.908+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.909+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.909+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.909+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.909+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.910+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.910+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.910+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.910+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.910+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.910+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.911+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.911+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.911+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.911+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.912+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.912+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.913+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.913+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.914+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.914+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.914+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.914+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.916+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.916+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.916+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.916+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.917+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.917+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.917+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.917+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.918+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.918+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.918+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.918+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.885+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.885+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.887+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.887+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.887+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.887+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.888+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.888+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.885+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.885+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.885+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.885+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.886+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.886+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.886+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.886+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.886+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.886+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.887+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.887+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.887+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.887+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.888+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.888+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.888+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.888+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.889+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.889+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.889+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.889+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.890+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.890+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.891+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.891+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.891+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.891+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.892+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.892+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.892+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.892+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.893+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.893+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.893+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.893+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.894+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.894+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.894+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.894+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.895+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.895+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.895+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.895+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.896+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.896+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.896+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.896+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.897+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.897+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.897+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.897+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.898+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.898+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.898+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.898+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.899+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.899+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.899+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.899+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.900+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.900+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.900+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.900+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.901+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.901+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.901+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.901+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.902+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.902+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.902+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.902+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.903+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.903+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.903+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.903+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.904+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.904+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.904+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.904+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.905+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.905+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.905+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.905+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.906+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.906+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.906+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.906+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.907+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.907+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.907+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.907+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.908+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.908+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.908+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.908+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.909+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.909+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.909+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.909+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.910+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.910+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.910+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.910+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.911+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.911+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.911+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.911+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.912+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.912+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.912+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.912+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.913+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.913+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.913+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.913+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.914+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.914+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.914+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.914+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.915+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.915+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.915+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.915+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.916+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.916+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.916+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.916+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.917+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.917+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.917+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.917+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.918+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.918+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.918+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.918+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.919+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.919+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.919+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.919+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.919+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.919+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.919+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.919+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.919+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.919+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.920+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.920+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.920+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.920+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.920+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.920+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.920+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.920+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.920+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.920+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.921+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.921+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.921+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.921+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.921+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.921+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.922+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.922+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.922+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.922+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.922+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.922+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.922+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.922+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.923+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.923+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.923+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.923+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.923+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.923+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.923+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.923+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.924+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.924+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.924+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.924+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.924+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.924+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.924+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.924+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.924+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.924+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.925+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.925+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.925+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.925+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.925+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.925+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.925+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.925+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.926+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.926+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.926+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.926+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.926+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.926+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.926+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.926+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.927+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.927+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.927+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.927+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.927+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.927+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.927+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.927+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.928+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.928+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.928+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.928+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.928+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.928+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.928+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.928+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.929+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.929+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.929+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.929+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.929+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.929+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.929+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.929+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.930+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.930+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.930+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.930+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.930+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.930+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.930+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.930+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.931+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.931+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.931+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.931+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.932+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.932+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.932+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.932+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.932+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.932+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.932+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.932+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.933+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.933+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.933+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.933+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.933+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.933+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.933+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.933+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.934+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.934+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.934+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.934+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.934+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.934+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.934+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.934+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.935+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.935+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.935+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.935+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.935+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.935+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.935+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.935+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.935+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.935+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.936+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.936+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.936+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.936+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.936+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.936+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.936+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.936+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.937+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.937+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.937+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.937+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.937+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.937+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.937+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.937+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.938+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.938+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.938+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.938+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.938+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.938+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.938+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.938+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.939+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.939+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.939+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.939+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.939+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.939+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.939+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.939+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.940+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.940+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.940+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.940+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.940+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.940+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.941+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.941+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.941+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.941+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.942+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.942+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.942+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.942+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.943+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.943+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.943+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.943+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.944+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.944+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.944+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.944+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.945+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.945+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.945+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.945+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.946+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.946+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.947+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.947+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.947+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.947+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.948+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.948+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.948+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.948+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.949+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.949+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.949+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.949+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.950+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.950+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.950+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.950+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.951+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.951+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.951+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.951+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.952+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.952+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.952+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.952+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.953+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.953+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.953+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.953+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.954+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.954+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.954+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.954+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.956+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.956+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.956+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.956+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.957+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.957+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.957+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.957+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.958+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.958+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.958+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.958+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.959+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.959+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.959+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.959+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.960+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.960+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.960+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.960+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.961+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.961+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.961+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.961+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.962+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.962+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.962+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.962+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.963+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.963+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.964+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.964+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.964+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.964+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.964+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.965+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.965+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.965+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.965+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.965+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.966+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.966+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.966+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.966+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.967+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.967+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.967+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.967+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.968+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.968+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.968+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.968+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.969+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.969+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.969+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.969+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.970+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.970+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.970+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.970+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.971+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.971+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.971+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.971+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.972+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.972+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.972+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.972+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.973+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.973+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.973+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.973+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.973+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.973+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.974+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.974+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.974+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.974+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.975+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.975+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.975+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.975+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.976+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.976+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.976+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.976+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.977+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.977+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.977+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.977+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.978+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.978+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.978+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.978+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.979+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.979+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.979+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.979+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.980+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.980+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.980+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.980+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.981+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.981+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.981+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.981+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.982+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.982+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.982+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.982+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.984+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.984+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.984+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.984+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.984+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.984+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.985+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.985+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.985+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.985+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.986+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.986+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.951+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.951+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.953+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.953+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.955+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.955+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.968+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.968+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.971+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.971+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.971+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.971+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.983+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.983+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.984+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.984+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.984+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.984+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.940+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.940+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.941+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.941+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.941+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.941+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.942+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.942+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.942+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.942+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.943+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.943+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.943+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.943+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.944+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.944+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.944+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.945+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.945+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.945+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.946+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.946+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.947+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.947+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.948+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.948+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.948+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.948+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.949+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.949+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.949+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.949+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.950+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.950+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.950+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.950+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.951+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.951+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.952+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.952+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.952+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.952+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.953+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.953+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.953+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.953+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.954+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.954+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.954+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.954+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.955+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.955+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.955+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.955+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.956+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.956+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.956+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.956+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.957+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.957+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.957+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.957+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.958+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.958+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.958+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.958+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.959+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.959+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.959+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.959+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.961+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.961+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.961+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.961+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.963+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.963+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.964+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.964+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.964+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.964+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.965+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.965+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.965+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.965+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.966+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.966+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.966+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.966+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.967+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.967+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.967+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.967+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.968+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.968+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.968+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.968+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.969+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.969+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.969+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.969+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.970+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.970+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.970+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.970+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.971+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.971+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.971+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.971+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.972+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.972+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.972+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.972+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.973+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.973+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.973+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.973+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.974+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.974+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.975+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.975+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.975+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.975+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.977+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.977+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.977+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.977+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.978+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.978+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.978+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.978+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.979+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.979+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.980+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.980+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.981+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.981+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.981+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.981+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.982+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.982+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.982+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.982+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.983+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.983+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.983+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.984+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.984+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.984+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.984+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.984+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.985+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.985+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.985+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.985+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.986+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.986+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.986+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.986+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.987+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.987+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.987+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.988+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.988+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.988+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.989+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.989+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.989+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.989+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.990+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.990+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.990+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.990+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.990+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.991+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.991+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.991+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.992+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.992+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.992+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.992+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.986+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.986+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.987+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.987+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.987+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.987+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.988+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.988+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.988+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.988+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.989+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.989+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.989+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.989+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.990+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.990+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.990+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.990+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.991+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.991+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.991+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.991+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.992+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.992+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.992+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.992+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.992+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.993+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.993+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.993+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.994+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.994+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.994+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.994+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.994+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.994+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.995+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.995+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.995+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.995+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.996+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.996+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.996+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.996+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.997+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.997+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.997+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.997+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.998+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.998+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.998+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.998+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.999+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.999+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.999+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:01.999+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.000+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.000+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.000+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.000+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.001+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.001+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.001+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.001+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.002+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.002+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.002+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.002+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.003+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.003+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.003+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.003+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.004+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.004+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.004+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.004+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.005+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.005+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.005+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.005+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.005+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.005+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.006+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.006+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.006+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.006+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.007+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.007+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.007+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.007+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.008+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.008+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.008+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.008+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.009+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.009+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.009+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.009+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.010+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.010+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.010+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.010+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.011+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.011+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.011+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.011+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.012+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.012+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.012+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.012+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.014+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.014+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.014+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.014+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.014+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.015+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.015+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.015+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.015+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.016+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.016+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.016+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.016+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.018+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.018+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.018+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.018+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.020+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.020+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.020+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.020+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.021+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.021+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.021+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.021+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.022+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.022+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.022+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.022+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.023+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.023+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.023+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.023+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.024+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.024+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.024+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.024+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.025+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.025+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.025+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.025+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.026+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.026+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.026+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.026+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.027+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.027+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.027+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.027+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.028+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.028+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.028+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.028+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.028+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.028+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.029+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.029+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.029+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.029+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.030+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.030+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.030+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.030+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.031+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.031+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.990+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.990+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.993+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.993+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.999+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:01.999+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.000+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.000+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.000+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.000+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.015+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.015+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.015+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.015+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.015+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.016+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.016+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.016+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.017+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.017+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.017+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.017+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.019+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.019+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.025+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.025+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.027+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.027+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.028+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.028+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.028+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.028+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.030+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.030+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.031+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.993+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.993+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.993+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.993+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.993+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.993+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.994+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.994+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.994+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.994+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.995+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.995+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.995+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.995+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.997+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.997+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.997+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.998+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.998+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.998+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.998+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.998+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.999+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.999+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.999+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:01.999+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.000+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.000+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.000+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.000+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.001+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.001+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.001+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.001+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.002+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.002+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.002+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.002+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.003+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.003+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.003+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.003+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.004+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.004+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.004+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.004+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.005+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.005+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.005+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.005+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.005+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.005+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.006+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.006+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.006+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.006+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.007+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.007+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.007+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.007+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.008+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.008+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.008+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.008+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.009+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.009+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.009+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.009+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.010+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.010+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.010+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.010+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.012+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.012+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.012+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.012+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.013+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.013+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.013+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.013+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.014+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.014+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.014+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.014+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.015+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.015+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.015+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.015+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.016+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.016+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.016+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.016+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.017+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.017+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.017+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.017+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.018+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.018+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.018+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.018+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.019+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.019+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.019+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.019+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.021+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.021+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.021+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.021+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.021+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.022+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.022+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.022+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.022+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.022+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.023+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.023+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.023+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.023+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.024+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.024+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.024+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.024+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.025+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.025+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.025+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.025+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.026+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.026+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.026+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.026+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.027+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.027+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.027+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.027+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.028+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.028+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.028+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.028+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.029+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.029+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.029+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.029+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.030+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.030+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.030+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.030+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.031+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.031+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.031+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.031+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.032+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.032+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.032+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.032+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.032+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.032+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.033+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.033+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.033+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.033+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.035+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.035+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.035+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.035+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.036+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.036+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.037+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.037+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.037+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.037+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.038+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.038+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.031+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.031+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.032+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.032+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.032+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.032+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.033+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.033+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.034+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.034+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.034+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.034+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.034+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.034+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.035+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.035+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.037+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.037+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.038+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.038+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.038+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.038+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.039+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.039+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.041+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.041+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.041+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.041+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.042+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.042+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.042+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.042+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.043+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.043+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.043+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.043+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.043+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.043+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.044+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.044+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.044+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.044+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.045+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.045+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.045+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.045+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.047+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.047+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.048+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.048+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.048+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.048+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.049+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.049+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.049+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.049+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.051+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.051+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.051+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.051+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.051+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.051+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.052+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.052+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.052+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.052+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.053+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.053+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.053+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.053+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.054+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.054+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.055+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.055+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.055+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.055+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.056+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.056+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.056+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.056+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.057+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.057+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.057+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.057+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.057+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.057+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.058+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.058+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.058+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.058+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.061+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.061+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.061+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.061+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.062+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.062+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.062+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.062+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.063+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.063+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.063+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.063+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.064+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.064+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.064+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.064+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.065+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.065+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.065+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.065+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.066+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.066+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.066+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.066+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.067+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.067+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.067+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.067+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.068+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.068+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.068+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.068+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.069+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.069+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.069+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.069+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.070+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.070+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.070+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.070+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.070+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.070+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.071+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.071+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.071+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.071+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.072+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.072+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.072+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.072+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.073+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.073+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.073+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.073+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.074+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.074+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.074+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.074+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.076+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.076+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.076+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.076+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.077+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.077+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.077+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.077+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.078+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.078+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.078+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.078+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.079+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.079+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.079+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.079+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.080+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.080+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.080+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.080+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.081+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.081+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.081+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.081+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.037+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.037+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.043+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.043+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.044+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.044+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.044+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.044+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.045+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.045+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.045+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.045+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.046+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.046+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.047+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.047+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.047+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.047+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.048+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.048+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.048+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.048+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.050+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.058+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.058+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.063+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.063+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.067+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.067+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.067+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.067+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.070+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.070+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.070+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.070+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.074+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.074+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.074+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.074+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.075+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.075+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.075+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.075+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.076+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.076+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.076+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.076+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.077+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.077+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.077+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.077+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.078+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.078+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.078+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.078+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.038+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.038+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.038+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.038+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.039+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.039+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.039+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.039+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.040+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.040+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.041+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.041+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.041+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.041+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.043+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.043+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.043+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.043+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.044+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.044+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.044+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.044+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.045+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.045+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.045+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.045+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.046+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.046+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.046+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.046+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.047+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.047+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.047+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.047+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.049+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.049+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.049+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.049+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.050+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.050+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.050+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.050+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.051+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.051+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.051+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.051+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.052+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.052+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.052+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.052+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.053+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.053+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.053+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.053+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.055+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.055+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.055+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.055+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.056+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.056+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.056+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.056+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.057+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.057+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.057+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.057+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.058+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.058+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.058+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.058+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.059+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.059+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.059+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.059+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.060+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.060+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.060+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.060+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.061+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.061+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.061+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.061+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.062+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.062+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.062+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.062+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.063+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.063+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.063+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.063+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.064+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.064+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.064+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.065+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.065+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.065+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.066+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.066+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.066+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.066+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.067+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.067+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.067+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.067+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.068+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.068+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.068+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.068+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.069+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.069+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.069+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.069+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.070+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.070+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.070+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.070+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.071+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.071+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.071+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.071+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.072+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.072+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.072+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.072+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.073+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.073+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.073+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.073+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.073+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.073+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.074+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.074+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.074+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.074+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.075+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.075+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.075+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.075+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.076+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.076+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.076+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.076+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.077+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.077+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.077+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.077+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.078+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.078+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.078+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.078+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.079+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.079+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.079+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.079+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.080+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.080+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.080+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.080+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.081+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.081+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.081+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.081+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.082+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.082+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.082+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.082+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.083+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.083+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.083+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.083+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.084+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.084+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.084+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.084+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.085+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.085+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.082+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.082+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.082+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.082+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.083+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.083+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.083+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.083+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.084+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.084+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.084+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.084+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.085+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.085+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.085+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.085+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.086+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.086+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.086+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.086+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.087+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.087+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.087+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.087+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.088+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.088+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.088+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.088+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.089+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.089+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.089+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.089+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.090+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.090+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.090+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.090+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.091+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.091+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.091+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.091+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.092+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.092+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.092+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.092+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.093+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.093+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.093+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.093+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.094+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.094+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.094+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.094+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.095+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.095+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.095+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.095+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.097+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.097+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.097+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.097+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.098+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.098+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.098+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.098+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.099+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.099+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.099+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.099+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.100+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.100+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.100+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.100+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.103+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.103+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.103+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.103+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.104+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.104+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.104+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.104+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.105+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.105+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.105+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.105+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.106+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.106+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.106+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.106+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.107+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.107+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.107+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.107+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.108+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.108+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.108+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.108+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.109+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.109+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.109+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.109+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.110+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.110+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.110+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.110+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.111+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.111+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.111+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.111+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.112+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.112+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.112+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.112+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.113+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.113+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.113+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.113+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.114+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.114+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.114+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.114+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.115+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.115+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.115+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.115+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.116+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.116+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.116+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.116+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.117+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.117+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.117+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.117+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.120+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.120+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.120+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.121+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.121+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.121+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.122+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.122+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.122+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.122+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.123+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.123+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.123+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.123+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.124+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.124+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.124+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.124+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.125+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.125+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.125+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.125+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.126+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.126+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.126+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.126+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.127+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.127+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.127+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.127+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.128+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.128+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.128+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.128+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.129+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.129+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.129+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.079+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.079+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.079+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.079+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.080+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.080+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.080+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.080+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.081+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.081+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.081+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.081+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.083+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.088+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.088+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.088+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.088+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.090+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.090+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.091+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.091+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.093+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.093+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.097+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.097+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.109+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.109+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.110+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.110+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.111+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.111+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.111+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.111+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.115+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.115+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.115+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.115+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.116+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.116+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.116+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.116+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.117+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.117+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.118+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.118+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.085+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.085+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.086+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.086+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.086+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.086+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.087+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.087+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.087+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.087+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.088+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.088+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.088+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.088+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.090+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.090+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.090+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.090+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.091+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.091+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.091+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.091+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.092+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.092+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.092+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.092+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.093+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.093+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.093+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.093+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.094+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.094+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.094+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.094+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.095+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.095+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.095+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.095+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.096+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.096+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.096+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.096+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.097+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.097+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.097+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.097+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.098+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.098+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.099+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.099+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.099+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.099+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.100+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.100+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.100+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.100+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.101+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.101+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.101+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.101+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.102+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.102+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.102+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.102+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.103+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.103+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.103+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.103+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.104+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.104+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.104+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.104+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.105+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.105+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.105+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.105+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.106+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.106+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.106+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.106+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.107+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.107+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.107+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.107+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.108+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.108+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.108+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.108+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.109+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.109+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.110+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.110+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.110+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.110+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.111+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.111+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.111+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.111+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.112+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.112+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.112+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.112+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.113+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.113+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.113+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.113+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.114+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.114+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.115+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.115+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.115+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.115+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.116+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.116+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.116+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.116+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.117+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.117+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.117+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.117+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.118+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.118+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.118+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.118+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.119+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.119+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.119+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.119+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.120+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.120+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.120+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.120+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.121+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.121+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.121+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.121+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.122+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.122+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.122+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.122+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.123+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.123+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.123+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.123+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.124+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.124+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.124+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.124+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.125+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.125+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.125+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.125+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.126+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.126+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.126+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.126+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.128+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.128+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.129+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.129+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.130+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.130+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.130+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.130+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.131+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.131+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.131+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.131+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.132+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.132+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.132+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.132+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.133+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.133+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.133+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.133+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.134+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.134+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.134+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.134+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.135+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.135+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.135+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.129+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.130+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.130+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.130+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.130+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.131+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.131+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.131+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.131+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.132+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.132+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.132+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.132+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.133+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.133+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.133+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.133+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.134+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.134+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.134+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.134+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.135+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.135+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.135+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.135+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.136+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.136+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.136+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.136+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.137+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.137+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.137+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.137+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.139+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.139+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.139+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.139+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.140+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.140+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.140+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.140+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.141+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.141+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.141+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.141+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.142+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.142+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.142+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.142+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.143+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.143+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.143+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.143+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.144+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.144+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.144+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.144+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.145+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.145+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.145+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.145+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.146+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.146+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.146+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.146+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.147+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.147+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.147+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.147+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.148+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.148+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.148+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.148+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.149+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.149+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.149+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.149+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.150+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.150+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.150+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.150+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.151+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.151+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.151+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.151+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.152+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.152+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.152+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.152+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.153+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.153+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.153+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.153+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.154+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.154+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.154+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.154+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.155+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.155+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.156+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.156+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.156+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.156+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.157+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.157+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.157+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.157+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.158+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.158+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.158+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.158+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.159+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.159+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.159+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.159+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.160+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.160+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.160+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.160+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.161+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.161+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.162+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.162+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.163+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.163+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.163+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.163+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.164+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.164+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.164+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.164+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.165+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.165+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.165+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.165+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.166+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.167+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.167+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.168+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.168+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.168+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.168+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.169+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.169+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.169+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.169+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.169+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.169+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.170+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.170+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.170+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.170+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.171+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.171+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.171+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.171+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.172+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.172+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.172+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.172+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.173+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.173+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.173+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.173+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.174+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.174+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.174+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.174+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.175+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.175+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.175+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.175+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.176+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.176+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.176+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.129+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.129+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.142+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.142+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.142+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.142+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.145+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.145+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.145+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.145+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.153+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.153+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.154+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.154+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.154+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.154+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.155+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.155+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.156+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.156+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.156+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.156+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.158+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.158+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.163+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.163+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.164+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.164+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.164+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.164+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.168+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.168+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.168+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.168+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.172+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.135+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.136+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.136+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.136+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.136+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.137+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.137+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.137+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.137+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.139+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.139+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.139+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.139+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.140+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.140+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.140+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.140+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.141+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.141+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.142+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.142+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.142+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.142+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.143+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.143+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.143+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.143+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.144+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.144+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.144+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.144+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.145+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.145+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.145+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.145+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.146+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.146+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.146+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.146+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.147+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.147+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.147+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.147+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.148+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.148+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.148+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.148+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.148+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.148+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.149+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.149+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.149+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.149+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.150+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.150+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.150+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.150+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.151+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.151+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.151+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.151+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.152+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.152+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.152+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.152+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.153+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.153+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.153+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.153+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.154+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.154+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.154+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.154+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.155+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.155+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.155+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.155+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.158+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.158+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.158+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.158+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.159+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.159+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.159+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.159+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.160+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.160+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.160+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.160+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.161+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.161+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.161+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.161+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.162+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.162+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.162+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.162+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.162+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.162+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.163+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.163+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.163+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.163+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.164+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.164+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.164+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.164+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.165+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.165+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.165+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.165+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.166+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.166+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.166+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.166+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.167+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.167+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.167+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.167+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.168+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.168+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.168+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.168+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.169+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.169+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.169+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.169+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.170+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.170+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.170+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.170+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.171+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.171+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.171+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.171+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.172+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.172+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.172+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.172+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.172+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.172+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.173+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.173+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.173+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.173+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.174+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.174+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.174+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.174+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.176+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.176+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.176+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.176+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.177+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.177+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.177+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.177+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.178+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.178+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.178+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.178+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.179+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.179+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.179+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.179+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.180+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.180+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.180+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.180+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.181+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.176+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.177+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.177+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.177+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.177+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.178+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.178+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.178+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.178+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.179+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.179+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.180+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.180+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.180+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.180+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.181+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.181+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.181+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.181+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.182+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.182+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.182+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.182+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.183+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.183+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.183+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.183+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.184+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.184+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.184+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.184+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.185+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.185+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.186+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.186+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.186+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.186+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.187+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.187+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.187+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.187+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.189+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.189+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.189+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.189+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.190+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.190+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.191+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.191+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.191+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.191+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.192+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.192+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.192+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.192+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.193+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.193+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.193+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.193+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.194+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.194+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.194+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.194+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.195+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.195+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.195+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.195+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.196+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.196+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.196+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.196+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.196+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.196+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.197+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.197+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.197+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.197+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.198+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.198+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.198+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.198+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.199+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.199+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.199+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.199+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.200+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.200+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.200+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.200+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.201+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.201+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.202+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.202+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.202+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.202+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.203+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.203+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.203+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.203+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.204+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.204+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.204+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.204+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.205+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.205+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.205+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.205+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.206+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.206+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.206+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.206+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.207+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.207+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.207+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.207+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.208+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.208+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.208+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.208+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.209+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.209+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.209+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.209+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.210+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.210+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.210+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.210+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.211+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.211+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.211+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.211+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.212+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.212+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.212+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.212+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.213+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.213+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.213+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.213+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.214+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.214+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.214+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.214+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.215+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.215+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.215+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.215+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.216+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.216+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.216+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.216+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.217+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.217+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.217+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.217+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.218+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.218+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.218+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.218+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.219+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.219+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.219+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.219+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.220+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.220+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.222+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.222+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.222+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.222+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.223+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.178+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.178+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.178+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.178+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.186+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.186+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.186+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.186+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.198+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.198+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.209+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.209+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.218+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.218+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.181+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.182+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.182+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.182+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.182+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.184+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.184+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.184+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.184+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.185+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.185+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.185+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.185+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.186+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.186+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.186+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.186+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.187+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.187+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.187+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.187+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.189+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.189+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.189+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.189+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.190+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.190+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.190+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.190+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.191+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.191+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.191+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.191+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.193+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.193+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.193+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.193+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.197+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.197+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.197+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.197+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.198+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.198+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.198+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.198+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.199+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.199+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.199+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.199+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.199+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.199+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.200+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.200+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.200+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.200+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.201+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.201+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.201+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.201+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.202+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.202+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.202+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.202+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.203+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.203+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.203+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.203+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.204+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.204+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.204+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.204+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.205+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.205+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.205+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.205+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.206+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.206+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.206+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.206+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.206+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.206+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.207+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.207+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.207+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.207+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.208+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.208+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.208+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.208+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.209+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.209+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.209+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.209+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.210+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.210+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.210+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.210+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.211+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.211+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.211+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.211+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.213+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.213+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.213+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.213+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.214+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.214+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.214+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.214+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.215+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.215+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.215+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.215+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.216+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.216+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.216+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.216+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.217+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.217+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.217+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.217+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.218+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.218+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.218+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.218+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.219+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.219+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.219+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.219+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.220+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.220+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.220+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.220+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.221+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.221+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.221+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.221+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.222+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.222+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.222+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.222+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.223+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.223+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.223+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.223+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.224+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.224+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.224+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.224+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.225+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.225+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.225+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.225+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.225+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.225+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.223+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.223+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.223+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.224+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.224+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.224+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.224+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.225+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.225+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.225+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.225+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.226+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.226+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.226+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.226+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.227+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.227+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.227+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.227+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.228+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.228+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.228+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.228+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.229+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.229+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.229+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.229+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.230+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.230+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.230+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.230+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:02.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.231+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:02.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.231+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.231+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.231+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.232+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.232+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.232+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.232+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.233+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.233+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.233+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.233+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.234+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.234+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.234+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.234+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.235+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.235+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.235+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.235+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.236+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.236+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.236+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.236+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.237+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.237+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.237+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.237+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.238+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.238+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.238+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.238+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.239+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.239+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.239+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.239+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.240+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.240+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.240+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.240+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.241+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.241+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.241+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.241+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.242+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.242+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.242+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.242+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.243+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.243+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.243+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.243+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.244+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.244+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.244+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.244+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.245+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.245+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.245+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.245+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.246+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.246+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.246+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.246+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.247+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.247+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.248+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.248+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.248+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.248+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.249+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.249+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.249+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.249+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.250+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.250+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.250+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.250+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.251+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.251+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.251+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.251+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.252+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.252+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.252+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.252+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.253+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.253+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.253+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.253+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.254+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.254+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.254+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.254+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.255+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.255+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.255+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.255+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.256+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.256+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.256+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.256+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.257+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.257+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.257+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.257+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.258+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.258+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.258+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.258+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.259+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.259+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.259+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.259+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.260+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.260+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.260+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.260+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.261+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.261+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.261+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.261+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.262+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.262+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.262+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.262+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.263+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.263+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.263+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.263+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.265+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.265+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.265+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.266+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.266+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.266+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.266+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.267+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.267+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.267+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.267+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.269+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.269+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.269+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.269+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.270+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.270+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.270+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.270+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.220+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.220+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.221+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.221+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.221+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.221+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.234+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.234+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.234+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.234+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.244+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.244+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.246+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.246+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.227+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.227+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.227+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.227+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.228+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.228+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.228+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.228+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.229+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.229+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.229+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.229+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.230+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.230+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.230+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.230+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.231+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.231+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.231+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.231+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.232+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.232+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.232+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.232+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.233+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.233+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.234+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.234+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.234+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.234+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.235+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.235+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.235+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.235+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.236+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.236+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.236+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.236+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.237+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.237+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.237+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.237+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.238+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.238+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.238+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.238+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.239+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.239+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.239+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.239+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.240+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.240+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.240+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.240+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.241+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.241+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.241+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.241+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.241+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.241+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.242+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.242+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.242+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.242+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.243+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.243+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.243+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.243+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.244+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.244+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.244+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.244+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.245+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.245+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.245+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.245+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.246+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.246+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.246+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.246+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.248+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.248+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.249+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.249+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.249+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.249+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.250+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.250+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.250+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.250+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.251+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.251+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.251+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.251+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.252+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.252+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.252+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.252+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.253+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.253+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.253+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.253+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.254+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.254+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.254+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.254+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.255+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.255+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.255+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.255+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.256+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.256+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.256+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.256+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.257+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.257+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.257+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.257+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.258+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.258+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.258+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.258+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.259+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.259+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.259+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.259+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.260+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.260+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.260+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.260+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.261+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.261+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.261+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.261+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.262+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.262+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.262+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.262+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.263+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.263+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.263+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.263+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.264+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.264+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.264+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.264+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.266+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.266+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.267+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.267+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.267+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.267+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.267+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.268+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.268+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.268+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.268+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.268+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.269+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.269+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.269+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.269+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.270+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.270+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.270+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.270+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.271+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.271+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.271+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.271+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.272+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.272+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.272+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.272+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.273+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.273+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.273+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.273+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.274+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.274+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.274+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.274+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.275+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.275+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.275+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.275+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.271+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.271+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.271+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.271+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.272+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.272+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.272+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.272+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.273+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.273+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.273+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.273+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.274+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.274+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.274+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.274+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.276+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.276+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.276+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.276+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.277+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.277+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.277+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.277+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.278+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.278+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.278+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.278+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.279+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.279+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.279+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.279+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.280+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.280+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.280+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.280+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.281+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.281+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.281+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.281+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.282+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.282+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.282+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.282+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.283+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.283+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.283+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.283+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.284+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.284+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.284+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.284+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.285+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.285+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.285+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.285+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.286+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.286+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.286+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.286+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.287+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.287+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.287+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.287+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.288+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.288+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.288+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.288+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.289+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.289+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.289+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.289+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.290+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.290+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.290+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.290+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.291+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.291+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.291+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.291+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.292+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.292+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.294+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.294+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.294+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.294+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.295+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.295+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.295+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.295+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.296+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.296+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.296+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.296+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.297+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.297+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.297+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.297+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.298+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.298+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.298+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.298+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.299+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.299+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.299+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.299+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.300+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.300+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.300+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.300+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.301+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.301+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.301+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.301+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.302+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.302+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.302+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.302+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.303+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.303+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.303+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.303+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.304+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.304+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.304+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.304+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.305+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.305+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.305+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.305+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.306+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.306+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.306+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.306+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.307+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.307+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.307+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.307+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.308+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.308+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.308+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.308+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.309+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.309+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.309+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.309+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.310+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.310+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.310+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.310+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.311+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.311+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.311+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.311+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.312+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.312+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.312+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.312+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.313+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.313+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.313+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.313+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.314+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.314+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.314+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.314+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.315+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.315+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.315+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.315+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.316+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.316+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.316+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.316+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.317+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.317+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.318+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.318+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.318+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.318+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.319+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.319+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.319+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.319+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.283+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.283+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.283+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.283+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.286+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.286+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.287+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.287+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.288+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.288+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.291+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.291+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.291+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.291+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.292+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.292+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.295+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.295+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.295+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.295+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.297+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.297+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.311+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.311+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.311+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.311+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.312+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.312+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.312+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.312+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.313+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.313+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.313+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.313+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.314+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.314+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.315+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.315+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.276+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.276+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.276+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.276+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.277+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.277+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.277+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.277+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.278+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.278+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.278+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.278+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.279+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.279+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.279+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.279+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.280+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.280+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.280+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.280+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.281+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.281+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.282+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.282+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.282+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.282+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.283+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.283+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.283+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.283+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.284+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.284+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.284+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.284+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.285+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.285+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.285+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.285+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.286+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.286+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.286+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.286+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.287+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.287+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.287+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.287+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.288+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.288+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.288+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.288+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.289+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.289+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.289+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.289+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.290+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.290+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.290+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.290+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.291+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.291+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.291+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.291+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.292+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.292+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.294+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.294+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.294+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.294+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.295+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.295+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.295+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.295+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.296+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.296+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.296+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.296+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.297+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.297+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.297+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.297+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.298+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.298+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.298+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.298+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.299+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.299+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.299+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.299+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.300+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.300+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.300+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.300+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.301+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.301+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.301+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.301+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.302+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.302+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.302+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.302+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.303+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.303+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.303+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.303+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.304+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.304+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.304+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.304+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.305+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.305+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.305+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.305+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.306+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.306+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.306+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.306+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.307+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.307+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.307+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.307+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.308+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.308+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.308+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.308+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.309+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.309+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.309+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.309+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.310+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.310+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.310+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.310+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.311+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.311+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.311+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.311+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.312+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.312+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.312+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.312+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.313+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.313+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.313+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.313+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.314+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.314+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.314+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.314+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.315+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.315+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.315+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.315+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.316+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.316+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.316+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.316+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.317+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.317+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.317+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.317+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.318+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.318+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.318+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.318+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.319+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.319+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.319+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.319+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.320+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.320+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.320+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.320+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.321+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.321+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.321+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.321+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.322+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.322+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.322+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.322+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.323+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.323+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.323+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.323+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.320+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.320+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.320+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.320+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.321+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.321+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.321+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.321+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.322+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.322+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.322+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.322+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.323+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.323+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.323+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.323+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.324+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.324+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.324+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.324+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.325+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.325+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.325+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.325+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.326+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.326+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.326+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.326+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.327+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.327+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.327+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.327+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.327+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.327+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.328+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.328+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.328+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.328+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.329+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.329+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.329+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.329+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.330+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.330+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.330+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.330+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.331+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.331+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.331+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.331+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.332+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.332+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.333+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.333+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.333+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.333+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.334+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.334+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.334+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.334+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.335+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.335+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.335+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.335+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.336+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.336+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.336+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.336+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.337+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.337+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.337+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.337+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.338+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.338+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.338+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.338+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.339+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.339+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.339+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.339+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.340+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.340+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.340+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.340+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.341+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.341+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.341+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.341+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.342+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.342+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.342+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.342+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.343+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.343+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.343+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.343+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.344+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.344+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.344+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.344+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.344+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.344+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.345+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.345+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.345+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.345+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.346+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.346+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.346+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.346+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.347+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.347+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.347+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.347+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.348+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.348+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.348+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.348+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.349+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.349+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.349+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.349+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.350+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.350+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.350+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.350+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.351+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.351+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.351+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.351+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.352+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.352+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.352+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.352+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.353+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.353+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.353+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.353+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.354+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.354+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.354+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.354+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.355+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.355+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.355+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.355+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.356+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.356+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.356+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.356+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.357+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.357+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.357+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.357+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.358+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.358+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.358+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.358+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.359+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.359+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.359+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.359+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.360+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.360+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.360+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.360+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.361+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.361+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.361+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.361+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.362+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.362+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.362+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.362+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.363+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.363+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.363+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.363+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.364+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.364+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.364+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.364+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.365+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.365+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.365+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.365+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.366+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.366+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.316+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.316+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.316+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.316+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.318+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.318+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.324+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.324+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.324+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.324+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.325+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.325+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.325+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.325+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.326+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.326+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.326+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.326+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.327+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.327+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.327+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.327+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.328+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.328+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.328+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.328+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.328+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.328+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.329+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.329+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.329+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.329+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.330+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.330+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.330+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.330+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.331+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.331+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.331+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.331+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.332+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.332+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.333+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.333+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.333+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.333+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.335+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.335+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.335+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.336+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.336+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.336+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.336+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.336+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.337+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.337+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.337+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.337+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.337+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.338+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.338+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.339+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.339+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.339+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.339+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.340+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.340+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.340+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.340+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.340+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.341+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.341+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.341+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.341+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.341+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.342+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.342+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.342+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.342+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.343+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.343+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.343+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.343+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.344+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.344+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.344+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.344+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.345+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.345+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.345+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.345+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.346+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.346+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.346+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.346+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.347+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.347+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.347+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.347+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.348+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.348+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.348+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.348+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.349+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.349+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.349+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.349+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.350+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.350+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.350+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.350+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.351+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.351+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.351+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.351+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.352+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.352+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.352+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.352+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.353+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.353+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.353+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.353+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.354+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.354+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.354+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.354+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.355+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.355+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.355+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.355+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.356+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.356+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.356+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.356+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.357+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.357+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.357+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.357+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.358+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.358+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.358+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.358+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.359+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.359+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.359+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.359+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.361+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.361+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.361+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.361+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.362+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.362+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.362+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.362+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.363+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.363+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.363+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.363+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.364+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.364+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.364+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.364+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.364+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.365+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.365+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.365+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.365+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.365+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.366+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.366+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.366+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.366+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.367+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.367+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.367+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.367+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.368+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.368+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.368+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.368+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.369+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.369+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.369+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.369+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.366+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.366+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.367+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.367+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.367+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.367+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.368+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.368+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.368+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.368+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.369+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.369+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.369+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.369+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.369+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.369+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.370+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.370+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.370+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.370+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.371+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.371+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.371+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.371+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.372+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.372+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.372+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.372+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.373+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.373+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.373+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.373+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.374+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.374+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.374+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.374+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.375+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.375+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.375+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.375+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.376+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.376+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.376+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.376+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.377+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.377+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.377+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.377+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.378+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.378+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.378+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.378+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.379+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.379+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.379+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.379+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.380+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.380+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.380+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.380+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.380+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.380+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.381+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.381+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.381+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.381+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.382+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.382+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.382+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.382+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.383+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.383+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.383+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.383+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.384+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.384+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.384+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.384+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.385+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.385+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.385+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.385+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.387+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.387+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.388+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.388+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.388+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.388+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.388+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.389+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.389+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.389+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.389+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.389+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.390+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.390+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.390+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.390+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.391+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.391+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.391+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.391+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.392+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.392+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.394+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.394+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.394+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.394+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.395+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.395+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.395+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.395+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.396+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.396+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.396+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.396+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.396+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.396+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.397+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.397+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.397+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.397+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.398+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.398+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.398+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.398+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.399+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.399+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.399+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.399+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.400+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.400+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.400+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.400+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.401+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.401+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.401+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.401+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.402+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.402+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.402+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.402+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.403+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.403+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.403+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.403+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.404+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.404+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.404+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.404+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.405+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.405+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.405+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.405+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.406+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.406+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.406+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.406+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.407+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.407+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.407+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.407+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.408+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.408+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.408+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.408+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.409+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.409+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.409+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.409+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.410+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.410+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.410+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.410+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.411+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.411+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.411+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.411+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.412+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.412+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.412+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.412+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.413+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.413+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.413+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.413+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.414+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.414+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.414+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.407+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.407+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.370+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.370+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.370+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.370+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.371+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.371+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.371+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.371+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.372+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.372+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.372+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.372+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.373+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.373+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.373+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.373+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.373+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.373+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.374+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.374+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.375+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.375+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.375+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.375+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.375+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.375+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.376+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.376+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.376+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.376+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.377+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.377+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.377+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.377+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.378+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.378+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.378+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.378+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.379+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.379+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.379+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.379+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.380+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.380+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.380+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.380+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.381+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.381+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.381+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.381+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.382+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.382+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.382+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.382+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.383+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.383+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.383+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.383+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.383+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.383+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.384+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.384+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.384+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.384+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.385+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.385+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.385+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.385+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.386+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.386+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.386+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.386+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.387+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.387+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.387+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.387+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.388+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.388+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.388+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.388+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.389+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.389+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.389+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.389+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.391+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.391+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.391+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.391+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.392+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.392+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.392+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.392+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.392+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.392+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.393+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.393+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.393+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.393+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.394+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.394+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.394+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.394+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.395+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.395+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.395+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.395+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.396+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.396+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.396+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.396+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.397+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.397+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.397+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.397+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.398+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.398+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.398+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.398+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.399+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.399+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.399+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.399+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.400+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.400+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.400+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.400+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.401+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.401+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.401+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.401+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.402+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.403+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.403+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.403+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.403+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.403+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.403+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.404+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.404+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.404+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.404+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.405+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.405+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.405+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.405+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.406+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.406+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.406+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.406+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.407+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.407+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.407+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.407+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.408+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.408+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.408+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.408+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.409+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.409+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.409+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.409+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.410+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.410+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.410+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.410+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.411+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.411+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.411+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.411+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.413+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.413+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.413+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.413+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.414+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.414+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.414+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.414+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.415+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.414+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.414+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.415+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.415+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.415+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.415+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.415+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.416+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.416+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.416+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.416+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.417+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.417+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.417+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.417+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.418+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.418+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.418+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.418+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.419+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.419+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.419+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.419+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.420+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.420+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.420+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.420+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.421+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.421+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.421+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.421+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.422+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.422+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.422+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.422+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.423+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.423+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.423+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.423+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.424+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.424+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.424+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.424+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.425+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.425+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.425+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.425+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.426+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.426+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.426+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.426+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.427+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.427+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.427+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.427+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.428+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.428+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.428+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.428+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.429+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.429+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.429+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.429+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.430+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.430+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.430+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.430+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.431+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.431+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.431+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.431+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.432+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.432+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.432+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.432+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.433+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.433+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.433+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.433+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.434+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.434+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.434+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.434+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.435+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.435+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.435+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.435+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.436+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.436+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.436+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.436+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.437+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.437+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.437+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.437+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.438+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.438+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.438+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.438+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.439+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.439+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.439+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.439+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.440+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.440+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.440+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.440+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.441+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.441+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.441+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.441+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.442+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.442+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.442+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.442+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.443+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.443+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.443+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.443+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.443+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.443+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.444+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.444+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.445+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.445+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.445+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.445+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.446+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.446+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.446+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.446+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.447+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.447+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.447+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.447+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.448+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.448+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.448+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.448+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.449+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.449+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.449+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.449+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.450+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.450+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.450+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.450+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.451+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.451+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.451+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.451+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.452+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.452+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.452+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.452+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.453+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.453+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.453+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.453+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.454+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.454+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.454+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.454+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.455+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.455+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.455+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.455+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.456+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.456+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.456+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.456+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.456+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.457+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.457+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.457+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.459+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.459+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.460+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.460+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.460+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.460+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.461+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.461+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.461+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.461+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.462+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.462+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.462+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.411+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.411+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.411+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.411+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.412+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.412+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.415+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.415+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.418+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.418+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.419+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.419+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.420+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.420+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.420+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.420+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.423+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.423+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.423+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.423+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.424+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.424+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.425+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.425+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.425+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.425+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.434+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.434+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.434+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.434+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.436+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.436+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.445+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.445+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.446+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.446+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.447+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.447+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.448+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.448+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.454+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.454+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.415+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.415+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.415+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.416+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.416+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.416+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.416+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.416+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.416+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.417+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.417+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.418+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.418+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.418+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.418+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.419+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.419+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.419+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.419+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.420+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.420+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.420+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.420+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.422+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.422+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.422+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.422+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.423+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.423+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.423+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.423+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.424+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.424+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.424+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.424+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.425+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.425+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.425+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.425+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.426+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.426+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.426+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.426+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.428+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.428+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.428+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.428+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.429+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.429+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.429+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.429+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.430+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.430+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.430+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.430+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.430+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.430+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.432+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.432+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.432+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.432+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.433+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.433+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.433+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.433+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.434+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.434+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.434+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.434+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.436+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.436+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.436+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.436+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.437+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.437+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.437+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.437+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.438+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.438+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.438+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.438+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.439+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.439+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.439+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.439+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.440+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.440+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.440+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.440+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.440+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.441+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.441+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.442+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.442+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.442+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.442+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.444+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.444+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.444+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.444+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.445+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.445+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.445+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.445+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.446+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.446+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.446+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.446+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.447+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.447+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.448+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.448+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.448+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.448+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.449+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.449+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.449+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.449+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.450+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.450+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.450+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.450+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.452+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.452+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.453+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.453+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.453+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.453+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.454+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.454+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.454+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.454+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.456+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.456+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.456+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.456+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.457+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.457+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.458+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.458+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.458+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.458+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.459+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.459+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.459+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.459+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.460+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.460+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.460+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.460+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.461+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.461+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.461+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.461+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.462+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.462+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.462+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.462+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.463+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.462+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.462+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.462+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.463+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.463+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.464+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.464+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.464+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.464+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.465+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.465+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.465+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.465+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.466+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.466+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.466+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.466+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.467+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.467+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.467+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.467+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.468+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.468+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.468+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.468+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.468+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.468+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.469+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.469+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.469+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.469+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.470+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.470+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.470+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.470+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.471+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.471+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.471+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.471+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.472+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.472+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.472+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.472+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.473+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.473+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.473+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.473+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.474+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.474+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.474+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.474+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.475+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.475+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.475+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.475+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.476+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.476+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.476+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.476+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.477+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.477+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.477+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.477+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.478+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.478+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.478+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.478+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.479+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.479+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.479+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.479+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.480+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.480+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.480+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.480+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.481+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.481+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.481+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.481+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.481+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.481+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.482+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.482+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.482+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.482+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.483+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.483+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.483+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.483+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.484+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.484+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.484+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.484+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.485+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.485+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.485+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.485+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.486+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.486+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.486+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.486+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.487+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.487+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.487+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.487+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.488+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.488+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.488+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.488+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.489+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.489+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.489+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.489+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.490+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.490+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.490+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.490+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.491+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.491+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.491+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.491+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.492+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.492+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.492+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.492+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.492+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.492+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.493+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.493+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.493+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.493+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.494+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.494+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.494+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.494+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.495+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.495+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.496+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.496+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.496+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.496+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.496+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.496+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.497+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.497+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.497+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.497+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.498+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.498+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.498+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.498+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.499+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.499+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.499+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.499+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.500+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.500+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.500+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.500+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.501+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.501+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.501+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.501+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.502+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.502+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.502+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.502+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.503+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.503+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.503+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.503+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.504+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.504+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.504+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.504+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.505+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.505+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.505+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.505+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.506+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.506+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.506+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.506+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.507+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.507+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.507+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.507+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.507+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.460+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.460+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.460+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.460+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.491+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.501+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.501+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.501+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.501+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.502+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.502+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.502+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.502+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.504+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.504+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.504+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.504+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.463+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.463+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.463+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.464+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.464+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.464+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.464+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.465+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.465+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.465+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.465+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.466+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.466+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.466+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.466+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.467+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.467+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.467+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.467+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.468+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.468+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.468+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.468+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.470+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.470+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.470+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.470+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.471+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.471+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.471+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.471+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.472+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.472+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.472+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.472+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.472+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.472+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.473+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.473+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.473+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.473+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.474+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.474+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.474+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.474+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.475+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.475+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.475+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.475+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.476+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.476+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.476+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.476+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.477+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.477+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.477+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.477+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.478+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.478+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.478+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.478+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.479+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.479+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.479+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.479+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.480+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.480+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.480+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.480+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.482+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.482+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.483+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.483+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.483+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.483+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.484+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.484+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.484+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.484+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.485+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.485+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.485+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.485+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.486+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.486+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.487+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.487+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.487+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.487+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.488+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.488+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.488+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.488+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.489+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.489+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.489+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.489+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.490+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.490+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.490+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.490+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.491+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.491+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.491+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.491+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.492+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.492+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.492+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.492+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.493+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.493+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.493+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.493+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.494+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.494+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.494+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.494+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.495+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.495+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.495+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.495+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.496+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.496+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.496+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.496+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.497+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.497+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.497+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.497+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.498+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.498+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.498+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.498+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.499+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.499+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.499+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.499+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.500+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.500+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.500+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.500+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.501+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.501+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.501+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.501+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.502+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.502+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.502+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.502+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.503+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.503+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.503+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.503+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.504+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.504+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.706+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.706+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.707+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.707+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.707+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.707+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.708+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.708+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.708+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.708+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.709+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.709+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.709+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.709+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.710+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.710+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.710+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.710+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.711+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.711+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.711+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.711+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.712+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.712+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.712+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.507+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.508+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.508+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.508+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.508+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.509+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.509+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.509+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.509+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.510+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.510+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.510+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.510+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.511+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.511+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.511+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.511+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.512+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.512+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.512+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.512+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.513+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.513+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.513+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.513+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.514+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.514+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.514+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.514+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.514+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.514+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.515+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.515+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.515+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.515+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.516+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.516+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.516+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.516+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.517+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.517+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.518+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.518+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.519+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.519+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.519+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.519+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.520+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.520+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.520+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.520+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.521+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.521+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.521+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.521+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.521+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.521+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.522+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.522+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.522+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.522+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.523+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.523+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.523+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.523+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.524+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.524+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.524+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.524+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.524+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.524+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.525+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.525+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.526+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.526+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.526+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.526+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.526+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.526+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.527+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.527+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.527+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.527+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.528+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.528+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.528+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.528+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.529+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.529+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.529+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.529+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.529+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.530+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.530+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.530+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.530+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.530+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.531+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.531+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.531+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.531+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.532+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.532+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.532+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.532+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.533+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.533+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.533+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.533+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.533+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.533+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.534+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.534+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.534+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.534+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.535+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.535+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.535+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.535+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.536+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.536+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.536+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.536+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.536+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.537+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.537+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.537+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.537+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.537+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.538+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.538+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.538+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.538+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.539+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.539+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.539+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.539+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.540+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.540+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.540+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.540+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.541+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.541+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.541+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.541+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.542+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.542+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.542+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.542+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.542+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.542+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.543+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.543+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.543+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.543+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.544+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.544+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.544+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.544+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.545+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.545+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.545+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.545+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.545+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.545+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.546+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.546+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.546+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.546+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.547+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.547+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.547+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.547+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.548+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.548+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.548+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.548+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.548+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.548+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.549+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.549+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.549+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.549+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.550+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.550+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.550+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.550+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.551+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.551+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.513+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.513+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.513+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.513+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.526+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.526+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.526+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.526+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.526+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.526+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.530+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.530+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.530+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.530+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.712+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.713+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.713+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.713+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.713+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.714+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.715+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.715+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.715+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.715+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.716+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.716+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.716+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.716+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.717+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.717+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.717+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.717+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.718+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.718+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.718+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.718+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.719+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.719+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.719+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.719+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.719+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.719+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.720+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.720+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.721+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.721+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.721+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.721+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.722+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.722+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.722+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.722+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.723+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.723+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.723+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.723+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.724+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.724+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.724+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.724+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.725+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.725+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.725+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.725+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.726+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.726+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.726+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.726+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.727+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.727+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.727+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.727+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.728+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.728+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.728+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.728+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.729+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.730+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.730+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.730+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.730+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.731+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.731+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.731+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.731+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.731+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.731+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.732+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.732+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.732+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.732+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.733+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.733+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.735+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.735+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.735+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.735+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.736+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.736+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.736+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.736+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.737+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.737+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.738+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.738+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.738+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.738+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.739+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.739+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.739+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.739+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.740+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.740+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.740+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.740+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.741+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.741+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.741+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.741+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.742+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.742+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.742+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.742+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.742+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.742+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.743+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.743+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.745+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.745+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.745+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.745+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.746+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.746+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.746+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.746+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.747+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.747+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.747+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.747+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.748+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.748+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.748+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.748+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.749+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.749+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.749+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.749+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.750+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.750+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.750+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.750+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.751+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.751+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.751+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.751+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.752+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.752+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.752+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.753+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.753+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.753+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.753+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.754+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.754+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.754+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.754+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.755+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.755+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.755+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.755+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.756+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.756+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.756+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.756+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.757+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.758+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.758+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.758+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.758+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.759+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.759+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.759+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.759+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.760+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.760+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.760+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.760+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.761+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.761+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.761+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.761+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.551+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.551+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.551+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.551+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.552+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.552+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.552+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.552+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.553+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.553+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.553+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.553+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.554+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.554+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.554+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.554+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.554+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.554+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.555+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.555+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.555+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.555+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.556+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.556+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.556+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.556+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.557+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.557+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.557+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.557+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.558+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.558+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.558+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.558+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.558+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.558+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.559+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.559+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.559+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.559+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.560+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.560+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.560+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.560+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.561+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.561+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.561+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.561+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.561+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.561+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.562+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.562+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.562+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.562+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.563+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.563+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.563+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.563+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.564+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.564+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.564+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.564+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.565+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.565+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.565+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.565+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.565+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.565+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.566+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.566+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.566+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.566+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.567+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.567+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.567+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.567+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.567+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.567+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.568+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.568+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.568+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.568+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.569+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.569+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.569+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.569+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.570+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.570+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.570+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.570+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.571+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.571+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.571+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.571+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.571+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.571+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.572+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.572+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.572+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.572+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.573+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.573+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.573+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.573+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.574+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.574+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.574+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.574+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.574+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.574+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.575+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.575+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.575+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.575+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.576+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.576+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.576+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.576+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.577+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.577+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.577+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.577+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.578+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.578+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.578+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.578+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.578+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.578+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.579+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.579+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.579+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.579+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.580+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.580+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.580+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.580+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.581+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.581+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.581+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.581+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.582+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.582+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.582+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.582+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.582+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.583+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.583+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.583+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.583+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.583+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.584+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.584+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.584+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.584+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.585+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.585+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.585+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.585+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.586+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.586+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.586+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.586+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.586+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.586+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.587+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.587+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.587+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.587+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.588+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.588+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.588+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.588+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.589+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.589+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.589+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.589+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.590+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.590+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.590+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.590+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.590+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.590+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.591+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.591+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.591+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.591+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.592+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.592+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.555+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.555+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.555+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.555+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.555+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.555+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.761+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.761+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.762+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.762+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.762+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.762+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.763+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.763+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.763+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.763+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.764+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.764+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.764+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.764+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.765+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.765+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.765+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.765+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.766+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.766+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.766+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.766+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.767+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.767+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.767+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.767+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.768+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.768+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.768+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.768+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.769+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.769+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.769+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.769+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.770+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.770+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.770+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.770+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.771+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.771+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.771+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.771+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.772+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.772+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.772+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.772+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.773+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.773+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.773+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.773+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.774+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.774+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.774+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.774+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.775+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.775+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.775+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.775+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.776+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.776+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.776+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.776+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.777+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.777+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.777+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.777+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.778+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.778+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.778+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.778+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.779+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.779+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.779+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.779+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.592+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.592+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.593+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.593+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.593+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.593+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.593+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.593+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.594+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.594+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.594+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.594+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.595+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.595+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.595+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.595+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.596+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.596+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.596+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.596+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.596+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.596+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.597+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.597+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.597+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.597+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.598+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.598+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.598+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.598+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.599+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.599+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.599+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.599+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.599+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.599+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.600+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.600+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.600+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.600+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.601+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.601+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.601+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.601+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.602+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.602+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.602+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.602+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.602+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.602+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.603+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.603+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.603+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.603+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.604+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.604+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.604+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.604+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.604+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.604+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.605+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.605+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.605+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.605+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.606+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.606+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.606+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.606+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.606+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.606+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.607+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.607+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.607+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.607+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.608+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.608+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.608+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.608+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.609+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.609+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.609+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.609+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.609+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.609+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.610+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.610+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.610+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.610+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.612+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.612+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.613+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.613+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.613+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.613+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.613+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.613+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.614+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.614+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.614+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.614+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.615+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.615+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.615+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.615+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.616+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.616+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.616+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.616+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.617+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.617+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.617+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.617+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.618+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.618+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.618+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.618+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.618+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.618+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.619+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.619+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.619+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.619+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.620+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.620+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.620+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.620+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.621+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.621+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.621+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.621+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.621+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.621+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.622+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.622+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.622+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.622+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.623+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.623+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.623+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.623+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.624+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.624+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.624+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.624+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.624+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.624+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.625+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.625+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.625+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.625+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.626+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.626+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.626+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.626+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.628+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.628+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.628+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.628+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.629+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.629+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.629+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.629+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.630+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.630+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.630+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.630+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.631+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.631+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.631+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.631+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.632+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.633+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.633+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.633+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.633+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.634+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.634+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.634+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.634+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.635+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.635+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.635+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.635+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.598+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.598+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.598+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.598+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.625+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.625+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.625+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.625+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.626+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.626+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.780+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.780+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.780+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.780+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.781+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.782+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.782+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.782+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.782+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.783+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.783+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.783+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.783+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.784+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.784+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.784+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.784+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.785+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.785+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.785+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.785+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.786+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.786+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.786+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.786+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.787+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.787+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.787+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.787+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.788+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.788+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.788+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.788+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.789+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.789+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.789+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.789+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.790+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.790+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.790+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.790+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.791+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.791+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.791+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.791+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.792+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.792+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.792+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.792+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.793+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.793+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.793+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.793+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.794+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.794+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.794+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.794+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.795+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.795+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.795+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.795+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.796+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.796+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.796+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.796+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.797+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.797+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.797+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.797+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.798+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.798+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.798+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.798+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.799+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.799+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.799+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.799+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.800+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.800+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.800+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.800+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.801+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.801+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.801+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.801+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.802+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.802+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.802+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.802+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.803+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.803+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.803+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.803+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.804+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.804+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.804+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.804+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.805+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.805+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.805+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.805+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.806+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.806+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.806+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.806+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.807+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.807+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.807+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.808+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.808+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.808+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.808+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.809+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.809+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.809+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.809+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.810+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.811+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.811+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.811+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.811+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.812+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.812+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.812+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.812+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.813+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.813+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.813+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.813+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.814+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.814+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.814+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.814+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.815+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.815+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.815+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.815+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.816+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.816+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.816+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.816+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.817+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.817+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.817+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.817+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.818+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.818+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.818+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.819+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.819+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.819+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.819+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.819+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.819+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.820+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.820+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.820+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.820+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.821+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.821+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.821+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.821+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.822+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.822+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.822+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.822+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.823+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.823+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.823+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.823+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.825+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.825+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.826+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.826+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.827+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.827+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.827+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.827+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.636+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.636+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.636+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.636+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.636+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.636+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.637+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.637+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.637+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.637+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.638+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.638+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.638+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.638+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.639+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.639+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.639+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.639+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.640+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.640+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.640+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.640+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.641+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.641+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.641+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.641+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.641+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.641+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.642+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.642+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.642+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.642+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.643+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.643+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.643+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.643+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.644+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.644+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.644+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.644+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.644+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.644+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.645+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.645+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.645+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.645+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.646+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.646+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.646+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.646+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.646+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.646+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.647+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.647+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.647+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.647+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.648+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.648+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.648+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.648+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.649+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.649+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.649+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.649+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.650+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.650+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.650+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.650+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.651+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.651+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.651+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.651+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.652+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.652+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.652+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.652+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.653+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.653+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.653+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.653+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.653+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.653+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.654+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.654+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.654+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.654+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.655+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.655+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.655+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.655+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.656+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.656+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.656+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.656+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.657+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.657+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.657+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.657+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.657+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.657+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.658+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.658+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.658+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.658+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.659+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.659+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.659+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.659+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.660+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.660+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.660+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.660+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.660+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.660+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.661+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.661+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.661+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.661+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.662+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.662+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.662+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.662+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.663+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.663+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.663+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.663+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.663+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.663+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.664+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.664+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.664+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.664+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.665+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.665+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.665+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.665+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.666+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.666+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.666+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.666+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.667+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.667+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.667+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.667+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.667+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.667+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.668+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.668+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.668+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.668+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.669+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.669+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.670+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.670+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.670+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.670+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.671+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.671+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.671+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.671+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.671+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.671+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.672+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.672+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.672+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.672+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.673+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.673+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.673+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.673+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.674+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.674+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.674+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.674+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.674+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.675+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.676+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.676+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.676+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.676+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.677+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.677+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.677+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.677+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.650+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.650+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.666+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.666+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.827+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.827+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.828+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.828+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.829+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.829+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.829+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.829+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.830+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.830+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.830+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.830+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.831+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.831+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.831+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.831+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.832+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.832+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.832+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.832+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.833+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.833+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.833+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.833+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.834+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.834+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.834+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.834+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.835+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.835+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.835+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.835+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.836+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.836+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.837+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.838+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.838+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.838+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.838+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.839+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.839+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.839+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.839+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.840+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.840+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.840+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.840+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.841+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.841+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.841+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.841+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.842+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.842+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.842+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.842+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.843+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.843+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.843+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.843+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.844+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.844+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.844+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.844+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.845+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.845+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.845+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.845+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.846+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.846+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.846+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.846+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.847+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.847+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.847+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.847+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.848+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.848+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.848+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.848+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.849+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.849+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.849+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.849+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.850+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.850+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.850+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.850+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.851+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.851+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.852+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.852+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.852+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.852+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.853+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.853+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.853+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.853+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.854+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.854+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.854+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.854+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.678+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.679+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.679+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.679+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.679+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.679+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.679+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.680+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.681+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.681+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.681+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.681+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.682+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.682+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.682+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.682+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.682+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.682+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.683+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.683+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.683+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.683+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.684+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.685+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.685+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.685+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.685+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.686+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.686+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.686+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.686+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.687+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.687+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.687+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.687+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.688+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.689+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.689+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.689+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.689+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.690+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.690+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.690+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.690+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.691+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.692+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.692+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.692+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.692+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.693+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.693+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.693+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.693+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.694+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.694+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.694+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.694+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.694+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.694+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.695+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.695+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.695+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.695+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.696+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.697+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.698+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.698+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.698+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.698+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.699+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.699+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.699+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.699+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.700+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.700+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.700+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.700+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.701+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.701+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.701+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.701+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.701+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.701+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.702+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.702+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.702+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.702+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.703+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.703+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.703+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.703+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.703+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.703+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.704+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.704+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.704+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.704+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.705+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.706+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.706+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.706+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.706+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.707+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.707+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.707+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.707+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.708+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.708+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.708+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.708+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.709+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.709+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.709+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.709+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.710+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.710+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.710+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.710+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.711+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.711+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.711+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.711+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.712+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.712+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.712+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.712+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.713+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.713+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.713+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.713+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.714+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.714+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.714+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.714+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.715+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.715+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.715+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.715+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.716+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.716+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.716+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.716+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.717+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.717+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.717+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.717+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.718+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.719+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.719+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.719+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.719+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.720+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.720+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.722+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.685+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.685+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.716+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.716+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.716+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.855+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.855+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.855+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.855+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.856+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.856+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.856+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.856+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.857+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.857+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.857+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.857+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.858+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.858+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.858+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.859+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.859+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.859+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.859+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.860+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.860+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.860+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.860+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.861+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.861+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.861+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.861+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.862+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.862+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.862+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.862+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.863+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.863+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.864+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.864+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.864+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.864+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.865+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.865+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.865+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.865+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.866+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.867+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.867+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.867+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.867+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.868+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.868+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.868+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.868+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.869+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.869+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.869+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.869+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.870+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.870+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.870+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.870+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.871+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.871+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.872+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.872+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.872+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.872+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.873+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.873+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.873+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.873+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.874+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.874+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.874+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.874+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.875+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.875+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.875+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.875+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.876+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.876+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.876+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.876+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.877+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.877+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.877+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.877+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.878+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.878+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.878+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.878+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.879+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.879+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.879+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.879+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.880+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.880+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.880+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.880+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.881+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.881+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.881+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.881+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.882+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.883+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.883+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.883+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.883+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.884+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.885+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.885+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.885+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.885+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.886+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.886+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.886+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.886+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.887+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.887+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.887+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.887+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.888+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.888+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.888+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.888+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.889+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.889+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.889+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.889+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.890+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.890+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.890+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.890+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.891+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.891+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.891+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.891+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.892+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.892+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.892+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.892+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.893+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.893+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.893+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.893+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.894+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.894+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.894+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.894+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.895+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.895+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.895+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.895+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.897+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.897+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.898+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.898+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.899+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.899+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.899+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.899+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.900+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.900+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.900+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.900+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.901+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.901+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.901+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.901+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.902+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.902+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.902+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.902+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.903+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.903+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.903+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.903+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.904+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.904+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.904+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.904+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.905+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.905+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.722+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.722+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.722+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.723+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.723+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.723+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.723+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.724+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.724+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.724+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.724+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.725+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.725+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.725+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.725+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.725+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.725+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.726+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.726+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.726+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.726+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.727+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.727+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.727+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.727+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.728+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.728+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.728+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.728+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.729+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.729+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.729+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.729+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.730+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.730+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.730+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.730+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.731+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.731+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.731+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.731+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.732+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.732+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.732+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.732+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.733+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.733+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.733+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.733+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.734+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.734+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.734+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.734+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.735+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.735+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.735+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.735+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.736+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.736+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.738+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.738+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.738+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.738+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.738+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.738+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.739+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.739+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.739+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.739+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.740+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.740+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.740+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.740+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.741+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.741+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.741+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.741+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.742+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.742+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.742+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.742+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.743+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.743+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.743+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.743+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.744+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.744+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.744+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.744+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.745+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.745+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.745+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.745+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.746+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.746+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.746+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.746+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.747+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.747+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.747+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.747+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.748+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.748+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.749+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.749+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.749+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.749+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.750+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.750+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.750+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.750+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.751+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.751+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.751+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.751+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.752+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.752+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.752+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.752+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.753+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.753+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.753+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.753+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.754+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.754+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.754+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.754+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.755+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.755+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.755+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.755+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.756+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.756+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.756+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.756+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.757+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.757+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.757+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.757+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.758+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.758+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.758+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.758+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.759+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.759+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.759+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.759+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.760+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.760+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.760+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.760+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.761+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.761+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.761+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.761+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.762+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.762+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.762+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.762+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.763+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.763+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.763+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.763+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.764+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.764+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.764+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.764+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.765+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.765+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.765+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.765+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.766+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.766+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.766+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.766+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.767+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.767+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.767+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.767+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.768+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.768+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.768+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.768+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.769+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.769+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.769+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.716+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.718+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.718+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.718+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.718+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.721+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.721+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.728+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.728+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.734+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.734+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.751+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.751+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.754+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.754+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.756+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.756+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.756+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.756+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.905+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.905+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.906+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.906+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.906+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.906+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.907+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.907+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.907+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.907+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.908+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.908+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.908+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.908+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.909+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.909+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.909+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.909+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.910+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.910+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.910+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.910+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.911+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.911+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.911+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.911+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.912+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.912+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.912+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.912+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.913+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.913+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.913+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.913+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.914+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.914+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.914+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.914+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.915+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.915+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.916+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.916+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.916+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.916+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.916+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.916+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.917+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.917+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.917+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.917+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.918+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.918+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.918+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.918+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.919+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.919+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.919+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.919+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.920+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.920+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.920+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.920+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.921+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.921+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.921+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.921+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.922+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.922+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.922+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.922+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.923+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.923+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.923+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.923+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.924+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.924+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.924+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.924+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.925+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.925+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.925+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.925+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.926+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.926+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.926+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.926+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.927+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.927+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.927+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.927+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.928+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.928+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.929+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.929+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.929+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.929+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.930+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.930+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.930+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.930+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.931+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.931+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.931+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.931+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.932+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.932+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.932+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.932+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.933+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.933+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.933+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.933+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.934+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.934+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.934+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.934+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.935+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.935+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.936+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.936+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.936+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.936+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.937+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.937+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.937+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.937+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.938+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.938+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.938+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.938+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.939+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.939+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.939+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.939+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.940+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.940+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.940+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.940+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.941+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.941+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.941+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.941+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.942+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.942+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.943+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.943+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.943+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.943+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.944+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.944+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.944+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.944+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.945+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.945+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.945+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.945+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.946+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.946+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.946+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.946+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.947+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.947+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.947+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.947+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.948+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.948+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.948+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.948+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.949+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.949+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.949+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.949+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.950+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.950+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.950+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.950+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.951+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.951+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.951+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.951+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.952+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.952+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.952+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.952+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.953+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.953+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.953+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.953+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.769+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.770+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.770+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.770+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.770+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.771+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.772+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.772+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.772+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.772+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.774+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.774+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.775+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.775+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.775+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.775+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.776+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.776+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.776+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.776+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.777+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.777+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.777+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.777+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.778+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.778+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.778+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.778+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.779+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.780+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.780+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.780+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.780+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.781+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.781+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.781+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.781+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.782+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.782+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.782+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.782+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.783+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.783+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.783+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.783+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.784+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.784+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.784+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.784+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.785+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.785+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.785+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.785+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.786+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.786+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.786+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.786+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.787+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.787+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.787+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.787+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.788+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.788+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.788+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.788+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.789+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.789+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.789+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.789+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.790+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.790+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.790+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.790+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.791+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.791+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.791+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.791+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.792+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.793+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.793+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.793+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.793+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.794+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.794+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.794+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.794+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.795+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.795+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.795+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.795+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.796+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.796+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.796+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.796+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.797+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.797+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.797+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.797+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.798+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.798+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.798+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.798+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.799+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.799+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.799+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.799+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.800+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.800+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.800+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.801+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.801+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.801+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.801+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.802+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.802+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.802+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.802+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.803+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.803+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.803+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.803+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.804+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.804+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.805+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.805+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.805+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.805+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.806+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.806+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.806+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.806+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.807+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.807+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.807+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.807+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.807+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.808+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.808+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.808+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.808+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.809+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.809+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.809+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.809+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.810+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.810+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.811+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.811+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.811+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.811+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.812+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.812+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.812+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.812+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.813+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.813+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.813+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.813+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.814+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.814+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.814+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.814+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.815+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.815+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.815+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.815+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.816+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.816+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.816+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.816+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.817+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.817+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.817+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.817+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.817+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.817+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.818+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.818+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.769+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.769+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.769+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.769+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.770+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.770+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.770+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.770+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.771+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.771+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.771+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.771+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.772+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.772+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.772+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.772+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.775+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.775+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.775+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.775+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.777+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.777+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.777+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.777+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.778+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.778+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.778+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.778+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.782+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.782+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.782+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.782+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.786+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.786+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.796+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.796+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.806+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.806+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.812+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.812+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.812+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.812+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Stopped 2024-04-23T21:59:03.478 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2024-04-23T21:59:03.489 DEBUG:tasks.rbd_mirror.cluster1.client.mirror.3:waiting for process to exit 2024-04-23T21:59:03.490 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T21:59:03.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.954+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.954+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.954+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.954+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.955+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.955+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.955+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.955+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.956+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.956+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.956+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.956+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.957+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.957+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.957+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.957+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.958+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.958+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.958+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.958+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.959+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.959+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.960+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.961+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.961+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.961+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.961+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.962+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.962+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.962+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.962+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.963+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.963+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.963+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.963+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.964+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.964+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.964+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.964+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.964+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.964+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.965+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.965+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.965+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.966+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.966+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.966+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.966+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.966+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.967+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.967+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.967+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.967+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.968+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.968+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.968+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.968+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.969+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.969+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.969+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.969+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.970+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.970+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.970+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.970+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.971+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.971+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.971+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.971+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.972+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.972+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.972+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.972+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.973+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.973+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.973+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.973+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.974+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.974+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.974+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.974+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.975+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.975+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.975+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.975+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.976+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.977+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.977+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.977+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.977+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.978+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.978+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.978+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.978+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.979+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.979+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.979+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.979+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.980+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.980+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.980+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.980+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.980+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.980+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.981+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.981+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.981+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.982+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.982+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.982+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.982+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.982+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.983+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.983+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.983+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.984+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.984+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.984+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.984+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.985+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.985+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.985+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.986+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.986+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.986+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.986+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.987+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.987+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.987+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.987+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.988+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.988+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.988+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.988+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.989+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.989+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.989+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.989+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.990+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.990+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.990+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.990+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.991+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.991+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.991+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.991+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.992+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.992+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.992+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.992+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.993+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.993+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.993+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.993+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.994+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.994+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.994+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.994+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.995+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.995+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.995+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.995+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.996+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.997+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.997+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.997+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.997+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.998+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.998+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.998+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.998+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.999+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.999+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.999+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:02.999+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.000+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.000+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.000+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.000+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.818+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.818+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.819+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.819+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.819+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.819+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.820+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.820+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.820+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.820+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.821+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.821+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.821+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.821+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.822+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.822+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.822+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.822+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.823+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.823+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.823+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.823+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.824+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.824+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.825+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.825+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.825+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.825+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.826+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.826+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.826+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.826+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.827+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.827+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.827+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.827+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.828+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.828+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.828+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.828+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.829+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.829+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.829+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.829+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.830+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.830+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.830+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.830+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.831+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.832+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.832+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.832+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.832+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.833+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.833+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.833+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.833+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.834+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.834+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.834+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.834+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.835+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.835+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.835+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.835+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.836+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.836+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.836+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.836+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.837+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.837+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.837+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.837+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.838+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.838+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.838+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.838+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.839+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.839+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.839+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.839+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.840+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.840+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.840+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.840+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.841+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.841+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.841+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.841+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.842+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.842+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.842+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.842+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.843+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.843+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.843+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.843+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.844+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.844+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.844+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.844+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.845+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.846+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.846+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.846+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.846+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.847+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.847+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.847+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.847+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.848+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.848+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.848+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.848+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.849+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.849+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.849+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.849+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.850+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.850+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.850+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.850+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.851+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.851+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.851+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.851+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.852+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.852+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.852+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.852+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.853+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.853+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.853+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.853+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.854+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.854+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.854+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.854+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.855+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.855+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.855+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.855+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.856+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.856+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.856+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.856+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.857+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.857+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.857+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.857+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.858+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.858+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.858+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.858+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.859+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.859+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.859+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.859+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.860+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.860+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.860+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.860+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.861+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.861+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.861+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.861+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.862+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.862+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.862+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.862+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.863+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.863+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.863+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.863+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.864+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.864+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.864+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.864+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.865+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.865+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.866+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.866+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.824+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.824+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.841+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.841+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.842+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.842+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.844+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.844+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.855+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.855+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.859+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.859+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.001+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.001+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.001+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.001+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.002+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.002+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.002+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.002+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.003+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.003+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.003+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.003+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.004+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.004+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.004+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.004+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.005+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.005+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.005+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.005+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.006+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.006+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.006+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.006+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.007+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.007+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.007+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.007+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.008+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.008+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.008+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.008+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.009+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.009+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.009+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.009+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.010+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.010+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.010+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.010+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.011+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.012+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.012+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.012+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.012+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.013+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.013+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.013+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.013+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.014+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.014+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.014+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.014+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.015+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.015+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.015+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.015+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.016+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.016+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.016+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.016+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.017+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.017+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.017+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.017+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.018+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.018+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.018+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.018+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.019+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.019+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.019+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.019+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.020+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.021+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.021+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.866+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.866+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.867+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.867+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.867+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.867+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.868+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.868+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.868+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.868+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.869+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.869+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.869+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.869+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.870+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.870+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.870+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.870+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.871+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.871+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.871+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.871+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.871+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.871+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.872+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.872+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.872+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.872+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.873+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.873+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.873+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.873+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.874+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.874+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.874+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.874+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.875+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.875+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.875+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.875+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.876+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.876+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.876+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.876+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.877+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.877+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.877+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.877+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.878+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.878+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.878+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.878+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.879+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.879+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.879+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.879+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.880+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.880+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.880+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.880+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.881+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.881+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.881+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.881+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.882+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.882+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.882+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.882+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.883+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.883+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.883+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.883+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.884+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.884+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.884+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.884+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.885+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.885+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.885+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.885+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.886+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.886+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.886+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.886+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.887+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.887+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.887+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.887+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.888+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.888+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.888+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.888+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.889+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.889+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.889+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.889+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.890+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.890+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.890+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.890+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.891+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.891+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.891+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.891+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.892+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.892+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.892+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.892+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.893+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.893+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.893+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.893+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.894+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.894+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.894+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.894+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.895+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.895+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.895+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.895+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.896+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.896+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.896+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.896+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.897+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.897+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.898+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.898+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.898+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.898+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.899+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.899+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.899+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.899+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.900+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.900+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.900+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.900+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.901+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.901+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.901+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.901+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.902+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.902+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.902+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.902+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.903+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.903+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.903+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.903+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.904+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.904+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.904+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.904+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.905+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.905+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.905+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.905+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.906+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.906+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.906+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.906+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.907+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.907+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.907+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.907+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.908+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.908+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.908+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.908+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.909+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.909+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.909+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.909+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.910+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.910+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.910+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.910+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.911+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.911+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.911+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.911+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.912+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.913+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.913+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.886+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.886+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.887+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.887+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.889+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.889+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.894+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.894+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.894+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.894+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.896+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.896+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.896+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.896+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.022+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.022+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.022+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.022+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.023+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.023+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.023+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.023+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.024+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.024+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.024+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.024+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.025+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.025+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.025+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.025+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.026+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.026+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.026+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.026+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.027+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.027+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.027+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.027+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.028+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.028+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.028+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.028+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.028+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.028+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.029+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.029+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.029+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.030+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.030+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.030+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.030+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.030+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.031+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.031+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.031+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.031+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.032+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.032+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.032+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.032+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.033+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.033+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.033+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.033+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.034+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.035+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.035+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.035+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.035+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.036+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.036+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.036+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.036+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.037+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.037+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.037+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.037+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.038+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.038+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.038+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.038+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.039+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.039+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.039+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.039+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.040+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.040+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.040+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.040+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.041+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.042+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.043+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.043+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.043+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.043+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.044+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.044+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.044+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.044+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.045+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.045+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.045+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.045+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.046+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.046+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.046+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.046+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.048+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.049+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.049+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.049+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.049+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.050+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.050+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.050+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.050+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.051+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.051+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.051+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.051+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.052+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.052+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.052+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.052+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.053+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.053+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.053+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.053+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.054+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.055+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.055+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.055+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.055+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.056+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.056+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.056+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.056+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.057+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.057+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.059+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.059+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.059+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.059+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.060+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.060+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.060+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.060+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.060+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.060+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.061+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.061+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.061+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.061+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.062+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.062+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.062+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.062+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.063+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.063+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.063+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.063+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.063+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.063+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.064+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.064+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.064+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.064+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.065+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.065+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.065+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.065+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.066+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.066+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.066+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.066+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.067+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.067+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.067+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.067+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.068+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.068+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.068+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.068+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.069+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.069+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.069+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.069+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.913+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.913+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.915+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.916+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.916+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.916+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.916+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.917+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.917+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.917+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.917+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.918+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.918+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.918+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.918+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.919+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.919+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.919+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.919+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.920+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.920+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.920+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.920+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.921+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.921+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.921+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.921+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.922+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.922+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.922+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.922+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.923+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.923+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.923+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.923+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.924+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.924+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.924+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.924+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.925+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.925+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.925+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.925+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.925+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.925+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.926+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.926+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.926+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.926+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.927+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.927+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.927+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.927+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.928+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.928+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.928+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.928+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.929+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.929+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.929+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.929+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.930+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.930+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.930+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.930+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.931+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.932+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.932+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.932+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.932+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.933+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.933+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.933+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.934+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.934+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.934+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.934+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.934+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.935+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.935+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.936+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.936+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.936+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.936+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.937+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.937+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.937+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.937+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.938+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.938+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.938+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.938+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.939+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.939+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.940+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.940+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.940+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.940+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.940+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.941+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.941+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.941+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.942+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.942+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.942+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.942+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.943+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.943+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.943+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.943+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.944+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.944+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.944+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.944+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.944+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.945+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.945+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.945+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.945+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.945+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.946+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.946+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.946+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.946+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.947+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.947+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.947+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.947+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.948+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.948+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.948+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.948+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.949+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.949+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.949+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.949+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.950+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.950+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.950+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.950+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.951+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.951+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.952+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.952+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.952+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.952+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.953+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.953+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.953+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.953+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.954+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.954+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.954+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.954+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.955+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.956+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.956+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.956+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.956+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.957+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.957+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.957+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.957+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.958+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.958+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.959+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.959+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.959+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.959+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.960+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.960+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.960+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.960+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.961+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.961+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.961+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.961+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.962+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.962+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.962+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.962+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.963+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.924+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.924+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.924+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.924+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.926+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.926+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.926+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.926+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.928+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.928+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.935+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.935+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.938+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.938+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.944+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.944+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.949+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.949+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.950+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.950+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.950+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.950+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.951+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.951+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.951+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.951+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.952+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.952+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.952+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.952+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.953+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.953+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.953+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.953+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.954+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.954+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.069+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.069+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.070+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.070+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.070+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.070+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.071+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.071+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.071+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.071+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.072+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.072+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.072+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.072+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.073+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.073+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.073+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.073+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.074+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.074+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.074+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.074+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.074+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.074+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.075+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.075+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.075+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.075+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.076+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.076+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.076+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.076+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.077+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.077+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.077+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.077+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.077+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.077+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.078+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.078+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.079+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.079+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.079+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.079+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.079+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.079+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.080+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.080+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.080+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.080+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.081+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.081+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.081+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.081+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.082+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.082+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.082+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.082+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.083+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.083+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.083+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.083+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.083+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.083+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.084+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.084+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.084+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.084+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.085+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.085+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.085+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.085+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.085+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.085+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.086+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.086+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.086+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.086+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.087+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.087+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.087+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.087+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.088+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.088+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.088+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.088+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.089+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.963+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.963+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.963+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.964+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.964+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.965+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.965+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.965+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.965+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.966+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.966+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.966+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.966+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.967+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.967+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.967+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.967+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.968+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.968+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.968+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.968+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.969+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.969+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.969+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.969+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.970+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.970+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.970+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.970+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.971+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.971+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.971+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.971+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.971+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.971+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.972+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.972+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.973+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.973+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.973+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.973+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.974+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.974+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.974+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.974+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.975+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.975+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.975+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.975+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.975+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.975+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.976+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.976+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.976+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.976+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.977+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.977+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.977+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.977+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.978+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.978+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.978+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.978+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.979+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.979+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.979+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.979+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.980+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.980+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.980+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.980+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.981+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.981+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.981+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.981+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.982+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.982+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.982+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.982+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.983+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.983+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.983+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.984+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.984+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.984+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.984+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.985+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.985+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.985+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.985+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.986+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.986+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.987+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.987+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.987+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.987+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.988+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.988+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.988+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.988+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.989+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.989+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.989+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.989+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.989+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.989+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.990+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.990+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.990+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.990+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.991+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.991+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.991+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.991+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.992+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.992+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.992+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.992+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.993+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.993+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.994+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.994+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.994+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.994+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.995+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.995+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.995+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.995+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.996+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.996+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.996+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.996+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.997+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.997+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.997+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.997+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.998+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.998+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.998+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.998+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.999+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.999+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.999+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:02.999+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.000+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.000+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.000+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.000+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.001+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.001+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.001+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.001+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.002+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.002+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.002+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.002+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.003+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.003+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.003+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.003+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.004+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.004+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.004+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.004+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.005+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.005+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.005+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.005+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.006+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.006+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.006+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.006+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.007+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.007+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.007+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.007+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.008+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.008+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.008+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.008+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.009+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.009+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.009+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.009+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.010+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.010+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.010+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.969+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.969+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.971+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.971+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.971+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.971+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.974+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.974+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.974+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.974+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.975+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.975+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.975+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.975+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.979+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.979+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.979+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.979+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.986+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.986+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.987+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.987+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.987+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.987+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.990+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.990+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.998+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.998+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:02.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.000+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.000+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.090+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.090+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.090+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.090+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.091+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.091+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.091+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.091+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.092+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.092+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.092+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.092+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.093+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.093+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.093+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.093+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.093+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.093+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.094+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.094+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.094+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.094+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.095+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.095+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.095+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.095+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.096+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.096+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.096+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.096+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.096+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.096+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.097+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.097+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.097+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.097+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.098+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.098+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.099+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.099+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.099+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.099+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.099+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.099+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.100+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.100+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.100+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.100+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.101+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.101+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.101+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.101+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.102+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.102+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.102+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.102+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.103+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.103+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.103+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.103+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.103+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.103+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.104+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.104+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.104+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.104+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.105+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.105+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.105+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.105+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.106+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.106+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.106+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.106+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.106+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.106+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.107+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.107+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.107+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.107+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.108+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.108+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.108+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.108+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.109+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.110+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.110+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.110+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.110+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.111+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.111+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.111+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.111+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.112+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.112+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.112+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.112+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.113+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.113+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.113+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.113+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.114+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.114+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.114+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.114+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.115+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.115+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.115+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.115+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.115+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.115+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.116+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.116+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.116+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.116+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.117+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.117+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.117+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.117+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.118+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.118+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.118+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.118+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.119+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.119+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.119+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.119+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.120+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.120+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.120+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.120+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.121+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.121+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.121+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.121+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.121+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.121+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.122+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.122+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.122+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.122+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.123+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.123+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.123+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.123+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.124+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.124+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.124+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.124+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.125+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.125+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.125+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.125+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.126+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.126+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.126+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.126+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.127+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.127+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.127+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.127+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.127+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.127+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.128+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.128+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.128+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.128+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.129+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.129+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.129+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.129+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.130+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.130+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.130+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.130+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.131+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.131+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.131+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.131+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.132+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.132+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.132+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.132+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.010+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.011+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.011+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.011+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.011+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.012+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.012+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.012+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.012+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.013+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.014+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.014+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.014+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.014+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.016+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.016+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.017+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.018+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.018+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.018+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.018+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.019+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.020+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.020+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.020+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.020+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.021+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.021+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.023+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.023+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.023+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.023+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.024+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.024+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.024+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.024+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.024+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.024+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.025+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.025+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.025+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.025+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.026+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.026+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.026+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.026+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.027+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.027+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.027+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.027+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.028+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.028+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.028+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.028+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.029+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.029+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.029+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.029+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.030+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.030+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.030+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.030+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.031+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.031+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.031+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.031+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.032+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.032+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.032+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.032+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.033+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.033+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.033+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.033+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.034+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.034+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.034+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.034+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.035+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.035+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.035+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.035+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.036+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.036+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.036+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.036+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.037+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.037+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.037+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.037+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.038+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.038+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.038+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.038+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.039+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.039+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.039+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.039+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.040+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.040+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.040+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.040+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.041+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.041+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.041+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.041+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.042+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.042+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.042+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.042+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.043+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.043+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.043+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.043+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.044+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.044+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.044+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.044+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.045+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.045+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.045+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.045+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.046+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.046+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.046+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.046+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.047+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.047+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.047+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.047+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.048+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.048+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.048+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.048+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.049+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.049+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.049+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.049+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.050+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.050+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.050+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.051+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.051+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.051+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.051+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.052+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.052+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.052+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.052+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.053+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.053+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.053+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.053+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.054+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.054+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.054+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.054+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.055+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.055+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.055+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.055+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.056+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.056+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.056+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.056+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.057+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.057+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.057+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.057+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.058+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.058+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.058+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.058+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.059+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.016+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.016+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.017+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.017+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.017+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.017+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.025+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.025+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.025+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.025+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.031+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.031+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.032+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.032+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.046+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.046+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.132+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.132+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.133+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.133+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.133+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.133+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.134+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.134+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.134+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.134+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.135+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.135+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.135+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.135+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.136+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.136+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.136+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.136+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.138+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.139+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.139+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.140+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.140+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.140+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.140+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.141+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.141+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.141+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.141+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.141+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.141+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.142+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.142+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.142+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.142+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.143+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.143+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.143+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.143+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.144+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.144+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.144+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.144+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.145+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.145+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.145+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.145+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.145+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.145+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.146+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.146+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.146+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.146+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.147+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.147+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.147+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.147+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.148+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.148+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.150+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.150+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.150+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.150+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.150+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.150+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.151+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.151+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.151+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.151+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.152+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.152+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.152+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.152+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.153+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.153+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.153+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.153+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.154+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.154+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.154+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.154+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.154+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.155+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.155+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.155+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.155+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.155+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.156+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.157+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.158+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.158+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.158+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.158+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.159+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.159+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.159+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.159+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.160+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.160+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.160+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.160+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.160+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.160+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.161+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.161+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.161+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.161+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.162+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.162+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.162+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.162+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.163+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.163+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.163+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.163+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.164+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.164+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.164+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.164+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.165+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.165+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.165+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.165+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.165+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.165+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.166+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.166+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.166+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.166+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.167+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.167+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.060+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.061+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.061+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.061+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.061+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.062+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.062+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.062+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.062+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.062+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.062+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.063+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.063+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.063+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.063+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.064+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.064+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.064+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.064+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.065+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.065+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.065+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.065+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.066+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.066+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.066+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.066+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.066+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.066+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.067+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.067+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.068+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.068+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.068+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.068+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.069+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.069+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.069+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.069+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.070+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.070+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.070+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.070+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.071+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.071+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.071+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.071+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.072+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.072+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.072+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.072+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.072+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.072+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.073+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.073+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.073+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.073+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.074+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.074+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.074+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.074+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.075+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.076+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.076+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.076+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.076+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.077+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.077+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.077+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.077+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.078+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.078+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.078+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.078+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.079+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.079+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.079+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.079+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.080+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.080+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.080+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.080+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.080+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.080+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.081+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.081+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.081+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.081+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.082+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.082+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.082+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.082+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.083+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.083+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.083+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.083+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.083+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.083+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.084+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.084+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.084+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.084+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.085+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.085+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.085+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.085+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.086+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.086+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.086+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.086+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.086+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.087+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.087+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.087+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.087+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.087+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.088+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.088+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.088+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.088+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.089+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.089+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.089+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.089+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.090+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.090+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.090+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.090+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.091+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.091+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.091+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.091+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.091+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.091+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.092+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.092+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.092+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.092+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.093+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.093+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.093+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.093+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.094+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.094+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.094+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.094+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.094+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.094+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.095+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.095+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.095+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.095+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.096+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.097+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.097+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.097+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.097+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.099+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.099+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.100+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.101+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.102+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.103+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.103+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.103+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.103+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.104+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.104+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.167+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.167+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.168+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.168+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.168+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.168+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.169+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.169+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.169+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.169+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.169+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.170+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.171+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.171+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.172+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.172+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.172+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.172+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.173+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.173+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.173+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.173+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.174+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.174+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.174+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.174+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.174+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.174+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.175+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.176+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.176+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.176+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.176+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.177+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.177+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.177+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.177+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.178+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.178+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.178+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.178+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.179+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.179+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.179+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.179+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.179+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.179+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.180+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.180+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.180+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.180+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.181+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.182+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.182+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.182+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.182+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.183+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.183+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.183+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.184+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.184+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.184+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.184+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.184+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.184+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.185+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.185+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.185+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.185+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.186+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.186+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.186+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.186+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.187+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.187+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.187+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.187+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.187+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.187+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.188+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.189+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.189+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.189+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.189+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.190+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.190+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.190+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.190+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.191+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.191+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.191+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.191+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.191+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.191+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.192+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.193+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.193+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.193+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.193+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.194+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.195+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.196+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.197+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.197+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.197+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.197+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.198+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.198+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.198+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.198+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.198+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.198+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.199+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.199+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.199+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.199+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.200+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.200+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.200+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.200+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.201+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.201+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.201+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.201+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.202+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.202+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.202+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.202+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.202+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.202+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.203+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.203+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.203+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.203+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.204+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.204+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.204+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.204+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.205+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.205+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.205+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.205+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.205+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.205+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.206+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.206+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.206+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.206+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.207+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.207+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.207+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.207+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.208+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.208+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.208+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.208+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.208+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.208+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.209+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.209+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.209+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.209+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.210+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.210+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.210+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.210+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.104+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.104+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.104+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.104+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.105+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.105+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.105+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.105+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.106+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.106+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.106+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.106+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.106+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.106+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.107+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.107+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.107+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.107+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.108+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.108+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.108+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.108+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.109+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.109+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.109+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.109+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.110+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.110+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.110+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.110+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.111+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.111+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.111+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.111+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.111+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.111+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.112+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.112+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.112+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.112+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.113+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.113+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.113+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.113+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.114+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.114+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.114+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.114+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.115+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.115+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.115+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.115+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.116+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.116+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.116+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.116+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.116+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.116+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.117+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.117+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.117+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.117+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.118+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.119+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.120+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.120+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.120+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.120+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.121+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.121+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.121+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.121+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.122+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.122+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.122+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.122+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.122+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.122+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.123+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.123+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.123+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.123+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.124+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.124+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.124+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.124+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.125+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.125+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.125+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.125+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.126+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.126+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.126+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.126+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.126+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.126+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.127+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.127+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.127+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.127+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.128+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.128+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.128+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.128+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.129+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.129+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.129+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.129+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.130+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.130+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.130+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.130+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.131+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.131+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.131+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.131+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.131+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.131+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.132+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.132+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.132+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.132+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.133+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.133+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.133+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.133+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.134+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.134+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.134+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.134+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.135+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.135+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.135+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.135+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.136+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.136+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.136+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.136+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.136+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.136+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.137+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.137+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.138+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.140+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.140+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.140+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.140+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.141+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.141+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.141+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.141+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.142+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.142+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.142+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.142+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.143+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.143+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.143+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.143+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.144+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.144+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.144+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.144+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.144+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.144+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.145+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.145+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.145+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.145+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.146+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.146+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.146+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.146+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.147+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.147+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.147+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.147+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.148+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.148+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.211+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.211+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.211+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.211+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.212+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.212+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.213+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.213+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.213+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.213+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.214+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.214+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.214+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.214+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.215+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.215+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.215+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.215+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.216+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.216+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.216+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.216+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.216+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.216+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.217+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.217+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.217+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.217+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.218+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.218+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.218+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.218+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.219+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.219+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.219+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.219+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.220+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.220+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.220+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.220+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.221+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.221+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.221+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.221+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.222+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.222+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.222+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.222+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.223+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.223+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.223+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.223+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.223+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.223+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.224+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.224+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.224+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.224+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.225+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.225+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.225+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.225+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.226+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.226+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.226+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.226+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.226+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.227+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.227+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.227+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.227+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.228+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.228+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.228+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.228+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.229+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.229+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.229+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.229+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.230+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.230+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.230+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.230+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.230+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.230+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.231+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.231+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.231+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.231+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.148+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.148+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.149+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.149+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.149+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.149+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.150+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.150+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.150+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.150+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.151+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.151+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.151+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.151+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.152+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.152+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.152+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.152+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.153+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.153+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.153+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.153+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.154+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.154+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.154+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.154+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.154+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.154+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.155+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.155+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.155+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.155+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.156+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.156+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.156+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.156+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.157+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.157+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.157+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.157+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.158+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.158+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.158+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.158+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.159+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.159+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.159+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.159+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.160+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.160+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.160+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.160+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.161+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.162+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.163+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.163+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.163+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.163+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.164+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.164+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.164+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.164+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.165+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.165+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.166+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.167+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.168+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.168+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.168+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.168+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.169+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.169+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.169+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.169+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.170+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.170+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.170+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.170+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.171+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.171+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.171+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.171+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.172+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.172+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.172+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.172+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.173+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.173+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.173+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.173+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.173+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.174+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.174+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.174+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.174+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.174+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.175+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.175+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.175+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.175+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.176+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.176+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.176+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.176+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.177+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.177+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.177+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.177+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.178+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.178+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.178+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.178+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.179+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.179+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.179+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.180+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.180+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.180+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.180+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.181+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.181+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.181+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.181+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.182+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.182+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.182+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.182+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.183+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.183+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.183+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.183+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.183+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.183+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.184+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.184+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.184+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.184+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.185+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.186+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.186+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.186+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.186+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.187+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.187+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.187+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.187+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.187+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.187+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.188+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.189+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.189+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.189+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.189+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.190+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.191+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.191+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.191+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.191+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.192+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.192+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.192+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.192+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.193+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.193+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.193+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.193+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.194+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.194+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.194+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.194+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.195+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.195+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.195+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.195+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.195+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.195+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.196+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.196+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.196+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.196+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.197+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.197+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.197+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.197+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.198+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.198+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.198+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.198+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.199+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.199+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.199+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.199+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.200+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.200+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.200+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.200+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.200+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.200+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.201+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.202+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.202+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.202+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.202+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.203+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.203+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.203+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.203+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.204+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.204+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.204+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.204+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.204+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.204+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.205+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.205+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.205+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.205+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.206+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.206+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.206+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.206+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.207+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.207+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.207+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.207+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.207+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.207+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.208+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.208+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.208+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.208+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.209+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.209+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.209+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.209+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.210+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.210+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.210+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.210+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.210+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.210+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.211+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.211+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.211+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.211+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.212+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.212+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.212+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.212+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.213+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.213+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.213+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.213+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.214+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.214+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.214+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.214+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.214+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.214+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.215+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.215+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.215+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.215+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.216+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.216+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.216+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.216+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.217+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.217+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.217+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.217+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.218+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.218+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.218+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.218+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.218+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.218+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.219+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.219+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.220+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.221+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.222+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.222+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.222+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.222+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.223+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.223+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.223+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.223+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.224+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.224+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.224+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.224+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.225+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.225+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.225+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.225+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.225+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.225+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.226+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.226+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.226+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.226+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.227+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.227+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.227+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.227+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.228+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.228+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.228+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.228+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.228+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.228+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.229+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.229+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.229+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.229+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.230+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.230+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.230+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.230+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.231+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.231+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.231+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.231+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.232+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.232+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.232+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.232+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.233+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.233+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.233+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.233+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.233+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.233+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.255+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.255+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.256+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.256+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.265+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.265+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.265+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.265+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.283+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.283+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.283+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.283+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.286+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.232+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.232+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.232+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.232+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.233+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.233+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.233+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.233+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.234+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.234+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.234+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.234+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.235+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.235+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.235+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.235+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.235+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.235+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.236+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.236+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.236+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.236+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.237+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.237+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.237+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.237+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.238+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.238+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.238+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.238+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.238+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.238+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.239+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.239+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.239+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.239+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.240+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.240+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.240+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.240+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.241+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.241+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.241+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.241+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.241+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.241+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.242+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.242+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.242+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.242+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.243+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.243+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.243+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.243+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.244+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.244+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.244+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.244+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.245+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.245+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.245+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.245+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.245+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.245+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.246+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.246+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.246+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.246+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.247+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.247+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.247+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.247+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.248+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.248+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.248+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.248+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.249+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.249+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.249+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.249+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.250+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.250+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.250+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.250+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.251+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.251+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.251+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.251+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.252+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.252+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.252+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.252+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.253+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.253+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.253+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.253+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.254+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.254+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.254+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.254+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.255+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.255+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.255+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.255+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.256+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.256+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.256+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.256+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.257+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.257+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.257+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.257+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.258+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.258+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.258+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.258+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.259+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.259+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.259+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.259+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.260+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.260+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.260+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.260+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.261+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.261+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.261+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.261+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.262+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.262+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.262+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.262+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.263+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.263+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.263+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.263+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.264+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.264+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.264+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.264+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.265+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.265+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.265+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.265+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.266+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.266+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.266+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.266+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.267+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.267+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.267+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.267+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.268+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.268+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.268+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.268+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.269+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.269+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.269+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.269+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.270+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.270+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.270+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.270+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.270+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.270+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.271+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.271+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.271+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.271+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.272+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.272+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.272+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.272+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.273+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.273+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.273+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.273+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.274+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.274+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.274+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.274+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.275+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.275+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.276+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.276+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.276+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.276+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.234+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.234+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.234+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.234+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.235+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.235+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.235+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.235+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.236+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.236+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.236+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.236+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.236+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.236+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.237+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.237+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.237+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.237+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.238+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.238+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.238+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.238+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.239+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.239+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.239+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.239+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.239+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.239+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.240+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.240+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.240+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.240+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.241+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.241+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.241+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.241+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.242+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.242+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.242+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.242+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.243+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.243+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.243+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.243+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.244+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.244+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.244+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.244+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.244+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.244+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.245+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.245+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.245+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.245+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.246+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.246+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.246+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.246+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.247+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.247+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.247+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.247+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.247+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.247+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.248+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.248+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.248+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.248+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.249+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.249+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.249+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.249+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.250+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.250+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.250+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.250+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.251+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.251+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.251+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.251+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.252+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.252+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.252+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.252+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.253+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.253+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.253+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.253+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.254+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.254+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.254+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.254+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.255+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.255+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.255+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.255+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.256+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.256+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.256+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.256+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.257+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.257+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.257+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.258+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.258+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.258+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.259+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.259+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.259+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.259+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.260+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.260+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.260+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.260+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.261+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.261+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.261+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.261+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.262+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.262+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.262+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.262+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.262+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.262+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.263+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.263+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.263+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.263+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.264+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.265+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.265+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.265+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.265+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.266+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.266+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.266+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.267+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.267+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.267+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.268+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.269+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.269+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.269+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.269+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.269+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.269+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.270+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.270+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.270+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.270+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.271+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.271+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.271+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.271+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.272+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.272+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.272+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.272+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.273+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.273+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.273+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.273+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.274+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.274+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.274+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.274+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.275+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.275+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.275+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.276+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.276+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.276+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.276+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.277+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.277+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.277+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.277+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.278+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.278+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.280+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.280+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.281+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.277+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.277+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.277+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.277+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.278+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.278+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.278+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.278+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.279+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.279+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.279+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.279+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.280+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.280+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.280+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.280+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.281+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.282+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.282+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.282+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.282+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.283+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.283+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.283+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.283+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.284+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.284+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.284+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.284+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.285+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.285+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.285+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.285+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.286+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.286+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.286+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.286+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.281+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.281+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.281+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.282+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.282+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.282+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.282+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.283+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.283+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.283+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.283+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.284+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.284+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.284+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.284+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.284+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.285+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.285+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.285+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.285+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.285+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.286+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.286+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.287+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.287+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.287+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.287+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.288+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.288+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.288+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.288+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.289+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.289+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.290+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.290+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.290+0000 7f99a2eb5640 -1 rbd::mirror::InstanceWatcher: 0x5619a9428380 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4637 object: (2) No such file or directory 2024-04-23T21:59:03.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.290+0000 7f99a2eb5640 -1 rbd::mirror::InstanceWatcher: 0x5619a9928000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4637 object: (2) No such file or directory 2024-04-23T21:59:03.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.290+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.290+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.290+0000 7f999c6a8640 -1 rbd::mirror::InstanceWatcher: 0x5619a9428380 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:59:03.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.290+0000 7f999cea9640 -1 rbd::mirror::InstanceWatcher: 0x5619a9928000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:59:03.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.291+0000 7f999669c640 -1 rbd::mirror::MirrorStatusUpdater 0x5619a94925a0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:03.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.291+0000 7f999669c640 -1 rbd::mirror::MirrorStatusUpdater 0x5619a94927e0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:03.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.291+0000 7f99a36b6640 -1 rbd::mirror::NamespaceReplayer: 0x5619a949e000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:59:03.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.291+0000 7f99a36b6640 -1 rbd::mirror::NamespaceReplayer: 0x5619a949e1a0 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:59:03.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.291+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.291+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.291+0000 7f999c6a8640 -1 rbd::mirror::MirrorStatusUpdater 0x5619a9492240 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.291+0000 7f99a36b6640 -1 rbd::mirror::NamespaceReplayer: 0x5619a949e000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.291+0000 7f999cea9640 -1 rbd::mirror::MirrorStatusUpdater 0x5619a9492360 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.291+0000 7f99a2eb5640 -1 rbd::mirror::NamespaceReplayer: 0x5619a949e1a0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.291+0000 7f99a36b6640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.291+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a9380c00 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.292+0000 7f99a2eb5640 -1 rbd::mirror::InstanceWatcher: 0x5619a9428000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4637 object: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.292+0000 7f999cea9640 -1 rbd::mirror::InstanceWatcher: 0x5619a9428000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.292+0000 7f9995e9b640 -1 rbd::mirror::MirrorStatusUpdater 0x5619a91dc6c0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.292+0000 7f99a2eb5640 -1 rbd::mirror::NamespaceReplayer: 0x5619a9c88000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.293+0000 7f999c6a8640 -1 rbd::mirror::MirrorStatusUpdater 0x5619a9159320 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.293+0000 7f99a36b6640 -1 rbd::mirror::NamespaceReplayer: 0x5619a9c88000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.296+0000 7f99a2eb5640 -1 rbd::mirror::LeaderWatcher: 0x5619a968d200 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.297+0000 7f99a2eb5640 -1 rbd::mirror::InstanceWatcher: 0x5619a9429c00 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4640 object: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.297+0000 7f998f68e640 -1 rbd::mirror::InstanceWatcher: 0x5619a9429c00 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.297+0000 7f9988e81640 -1 rbd::mirror::MirrorStatusUpdater 0x5619a9492900 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.297+0000 7f99a2eb5640 -1 rbd::mirror::NamespaceReplayer: 0x5619a97c41a0 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:59:03.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.297+0000 7f998ee8d640 -1 rbd::mirror::MirrorStatusUpdater 0x5619a9643680 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:03.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:03.297+0000 7f99a36b6640 -1 rbd::mirror::NamespaceReplayer: 0x5619a97c41a0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:59:03.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.292+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.292+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.292+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.292+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.295+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.295+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.295+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.295+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.297+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.302+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.302+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.304+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.304+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.308+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.308+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.311+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.311+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.311+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.311+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.312+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.312+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.312+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.312+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.313+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.313+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.287+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.287+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.287+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.287+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.288+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.288+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.288+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.288+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.289+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.289+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.290+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.290+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.290+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.290+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.291+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.291+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.291+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.291+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.292+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.292+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.294+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.294+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.295+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.295+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.295+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.295+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.296+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.296+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.296+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.296+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.297+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.297+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.298+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.298+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.298+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.298+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.299+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.299+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.299+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.299+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.300+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.300+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.300+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.300+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.301+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.301+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.301+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.301+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.302+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.302+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.302+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.302+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.303+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.303+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.303+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.303+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.304+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.304+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.304+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.304+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.304+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.304+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.305+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.305+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.305+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.305+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.306+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.306+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.306+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.306+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.307+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.307+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.307+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.307+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.308+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.308+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.309+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.309+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.309+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.309+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.310+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.310+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.310+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.310+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.311+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.311+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.311+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.311+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.311+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.311+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.312+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.312+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.312+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.312+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.313+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.313+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.313+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.313+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.314+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.314+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.314+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.314+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.315+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.315+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.315+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.315+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.316+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.316+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.316+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.316+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.316+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.316+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.317+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.317+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.317+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.317+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.318+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.318+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.320+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.320+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.320+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.320+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.321+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.321+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.321+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.321+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.322+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.322+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.322+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.322+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.322+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.322+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.323+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.323+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.323+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.323+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.324+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.324+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.324+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.324+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.325+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.325+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.325+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.325+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.326+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.326+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.326+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.326+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.327+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.327+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.327+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.327+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.327+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.328+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.328+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.328+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.328+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.328+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.329+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.329+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.329+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.329+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.330+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.330+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.330+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.330+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.331+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.331+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.331+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.331+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.331+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.331+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.332+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.332+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.332+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.332+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.333+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.333+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.333+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.333+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.334+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.334+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.334+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.334+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.334+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.335+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.335+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.337+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.337+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.337+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.337+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.344+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.344+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.344+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.344+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.345+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.345+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.345+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.345+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.348+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.348+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.348+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.348+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.335+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.335+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.336+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.336+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.336+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.336+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.337+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.337+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.337+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.337+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.338+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.338+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.338+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.338+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.338+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.338+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.339+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.339+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.339+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.339+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.340+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.340+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.340+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.340+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.341+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.341+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.341+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.341+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.342+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.342+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.342+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.342+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.342+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.342+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.343+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.343+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.343+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.343+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.344+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.344+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.344+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.344+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.345+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.345+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.345+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.345+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.346+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.346+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.346+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.346+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.347+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.347+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.347+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.347+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.348+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.348+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.348+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.348+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.349+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.349+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.349+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.349+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.349+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.349+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.350+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.350+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.350+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.350+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.351+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.351+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.351+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.351+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.352+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.352+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.352+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.352+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.353+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.353+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.353+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.353+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.354+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.354+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.354+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.354+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.355+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.355+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.355+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.355+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.356+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.356+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.356+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.356+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.356+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.356+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.357+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.357+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.357+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.357+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.358+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.358+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.358+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.358+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.359+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.359+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.359+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.359+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.360+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.360+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.360+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.361+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.361+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.361+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.361+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.362+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.362+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.362+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.362+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.363+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.363+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.363+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.363+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.364+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.364+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.364+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.364+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.365+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.365+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.365+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.365+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.366+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.366+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.366+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.366+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.366+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.366+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.367+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.367+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.367+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.367+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.368+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.368+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.368+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.368+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.369+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.369+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.369+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.369+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.370+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.370+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.370+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.370+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.371+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.371+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.371+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.371+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.372+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.372+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.372+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.372+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.373+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.373+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.373+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.373+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.373+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.373+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.374+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.374+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.374+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.374+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.375+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.375+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.375+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.375+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.376+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.376+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.376+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.376+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.377+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.377+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.377+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.377+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.378+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.378+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.378+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.378+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.379+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.379+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.386+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.386+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.389+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.389+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.407+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.407+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.407+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.407+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.412+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.412+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.413+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.413+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.413+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.413+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.415+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.415+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.415+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.415+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.420+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.420+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.379+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.379+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.379+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.379+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.380+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.380+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.380+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.380+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.381+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.381+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.381+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.381+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.382+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.382+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.382+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.382+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.384+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.384+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.384+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.384+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.385+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.385+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.385+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.385+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.386+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.386+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.386+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.386+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.387+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.387+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.387+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.387+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.388+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.388+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.388+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.388+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.389+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.389+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.389+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.389+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.390+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.391+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.391+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.391+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.391+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.392+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.392+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.392+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.392+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.392+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.392+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.393+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.393+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.393+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.393+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.394+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.394+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.394+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.394+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.395+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.395+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.397+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.397+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.397+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.397+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.398+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.398+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.398+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.398+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.398+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.398+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.399+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.399+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.399+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.399+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.400+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.400+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.400+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.400+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.401+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.401+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.401+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.401+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.402+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.402+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.402+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.402+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.403+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.403+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.403+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.403+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.404+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.404+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.404+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.404+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.405+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.405+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.405+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.405+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.406+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.406+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.406+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.406+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.407+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.407+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.407+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.407+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.407+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.407+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.408+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.408+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.408+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.408+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.409+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.409+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.409+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.409+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.410+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.410+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.410+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.410+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.411+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.411+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.411+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.411+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.411+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.411+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.412+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.413+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.413+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.413+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.413+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.414+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.414+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.414+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.414+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.415+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.415+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.415+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.415+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.416+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.416+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.416+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.416+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.417+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.418+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.418+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.418+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.418+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.419+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.419+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.419+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.419+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.420+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.420+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.420+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.420+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.421+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.421+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.421+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.422+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.422+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.422+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.422+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.422+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.422+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.423+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.423+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.423+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.423+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.424+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.424+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.424+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.424+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.425+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.425+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.423+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.423+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.434+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.434+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.434+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.434+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.435+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.435+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.435+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.435+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.435+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.435+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.436+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.436+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.436+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.436+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.439+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.439+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.439+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.439+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.425+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.425+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.426+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.426+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.426+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.426+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.427+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.427+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.427+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.428+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.428+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.428+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.428+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.429+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.429+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.430+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.430+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.430+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.430+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.431+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.432+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.432+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.432+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.432+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.433+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.433+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.433+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.433+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.434+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.434+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.434+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.434+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.435+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.435+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.435+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.436+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.436+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.436+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.436+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.437+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.437+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.437+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.437+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.438+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.438+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.438+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.438+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.439+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.439+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.439+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.439+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.439+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.439+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.440+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.440+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.440+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.440+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.441+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.441+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.441+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.442+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.442+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.442+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.442+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.443+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.444+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.444+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.444+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.444+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.445+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.445+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.445+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.445+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.446+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.446+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.446+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.446+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.446+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.446+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.447+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.447+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.447+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.447+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.448+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.448+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.448+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.448+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.449+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.449+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.449+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.449+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.450+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.450+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.450+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.450+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.451+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.451+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.451+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.451+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.452+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.452+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.452+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.452+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.452+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.452+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.453+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.453+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.453+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.453+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.454+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.454+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.454+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.454+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.455+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.456+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.456+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.456+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.456+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.456+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.457+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.457+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.457+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.457+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.457+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.458+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.458+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.458+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.458+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.459+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.459+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.459+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.459+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.460+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.460+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.460+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.460+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.460+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.460+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.460+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.460+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.461+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.461+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.461+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.461+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.462+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.462+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.462+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.462+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.463+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.463+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.463+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.463+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.464+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.464+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.464+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.464+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.464+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.464+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.465+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.465+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.465+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.465+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.466+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.466+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.466+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.466+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.467+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.467+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.467+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.467+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.468+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.468+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.468+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.468+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.469+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.470+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.470+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.470+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.470+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.470+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.470+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.471+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.471+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.471+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.471+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.472+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.472+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.472+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.472+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.473+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.473+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.473+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.473+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.473+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.474+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.474+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.474+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.474+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.474+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.475+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.475+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.475+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.475+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.476+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.476+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.476+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.476+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.477+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.477+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.477+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.477+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.477+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.477+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.478+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.478+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.478+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.478+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.479+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.479+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.479+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.479+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.480+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.480+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.480+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.480+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.480+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.480+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.481+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.481+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.481+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.481+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.482+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.482+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.482+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.482+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.483+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.483+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.483+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.483+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.484+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.484+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.486+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.487+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.487+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.487+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.487+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.487+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.487+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.488+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.488+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.488+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.489+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.489+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.489+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.490+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.490+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.490+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.490+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.490+0000 7fee1a964640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 103838) UID: 1000 2024-04-23T21:59:03.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fee1a163640 -1 rbd::mirror::InstanceWatcher: 0x563edccfa000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5372 object: (2) No such file or directory 2024-04-23T21:59:03.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fee1a163640 -1 rbd::mirror::InstanceWatcher: 0x563edc808380 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5372 object: (2) No such file or directory 2024-04-23T21:59:03.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fee0613b640 -1 rbd::mirror::InstanceWatcher: 0x563edc809c00 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:59:03.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fee13956640 -1 rbd::mirror::InstanceWatcher: 0x563edccfa000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:59:03.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fee13956640 -1 rbd::mirror::InstanceWatcher: 0x563edc808380 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:59:03.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fee0c948640 -1 rbd::mirror::MirrorStatusUpdater 0x563edca46900 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:03.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fee0c948640 -1 rbd::mirror::MirrorStatusUpdater 0x563edc858360 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:03.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fee1a163640 -1 rbd::mirror::NamespaceReplayer: 0x563edc856000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:59:03.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.491+0000 7fee1a163640 -1 rbd::mirror::NamespaceReplayer: 0x563edc8561a0 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:59:03.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.492+0000 7fee13956640 -1 rbd::mirror::MirrorStatusUpdater 0x563edc858000 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:03.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.492+0000 7fee13956640 -1 rbd::mirror::MirrorStatusUpdater 0x563edc858120 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:03.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.492+0000 7fee19962640 -1 rbd::mirror::NamespaceReplayer: 0x563edc856000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:59:03.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.492+0000 7fee19962640 -1 rbd::mirror::NamespaceReplayer: 0x563edc8561a0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:59:03.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.492+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.492+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.492+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.492+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.492+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.492+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.492+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.492+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.493+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.493+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.493+0000 7fee19962640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.493+0000 7fee1a163640 -1 rbd::mirror::LeaderWatcher: 0x563edc76ac00 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (2) No such file or directory 2024-04-23T21:59:03.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.494+0000 7fee19962640 -1 rbd::mirror::InstanceWatcher: 0x563edc808000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5372 object: (2) No such file or directory 2024-04-23T21:59:03.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.494+0000 7fee13155640 -1 rbd::mirror::InstanceWatcher: 0x563edc808000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:59:03.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.494+0000 7fee0d149640 -1 rbd::mirror::MirrorStatusUpdater 0x563edc5cc6c0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:03.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.494+0000 7fee19962640 -1 rbd::mirror::NamespaceReplayer: 0x563edc4361a0 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:59:03.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.494+0000 7fee13956640 -1 rbd::mirror::MirrorStatusUpdater 0x563edc569440 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:03.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi150.stderr:2024-04-23T21:59:03.494+0000 7fee1a163640 -1 rbd::mirror::NamespaceReplayer: 0x563edc4361a0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:59:03.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.504+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.504+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:03.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:03.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.513+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.513+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.513+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.513+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.523+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.523+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.523+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.523+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.524+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.524+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.562+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.562+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.562+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.562+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.567+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.567+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.567+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.567+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.567+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.584+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.584+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.585+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.585+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.587+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.587+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.608+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.608+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.608+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.608+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.608+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.608+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.642+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.642+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.684+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.684+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.684+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.684+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.684+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.684+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.685+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.685+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.685+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.685+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.685+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.685+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.698+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.698+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.698+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.698+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.703+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.703+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.703+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.703+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.704+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.704+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.704+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.704+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.716+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.716+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.718+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.718+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.718+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.718+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.719+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.719+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.721+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.721+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.734+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.734+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.756+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.756+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.756+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.756+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.756+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.756+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.777+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.777+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.777+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.777+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.777+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.777+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.778+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.778+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.795+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.795+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.796+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.796+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.811+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.811+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.811+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.811+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.832+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.832+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.832+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.832+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.850+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.850+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.850+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.850+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.858+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.858+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.858+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.858+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.858+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.858+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.859+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.859+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.859+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.859+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.870+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.870+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.870+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.870+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.896+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.896+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.896+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.896+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.920+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.920+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.920+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.920+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.982+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.982+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:03.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.015+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.015+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.015+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.015+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.015+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.015+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.016+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.016+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.016+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.016+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.017+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.017+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.017+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.017+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.018+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.018+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.018+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.018+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.018+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.018+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.019+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.019+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.019+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.019+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.020+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.025+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.025+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.025+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.025+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.025+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.025+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.027+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.027+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.027+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.027+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.027+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.027+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.028+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.028+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.028+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.028+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.030+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.030+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.030+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.030+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.031+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.031+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.031+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.031+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.031+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.031+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.032+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.032+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.032+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.032+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.063+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.063+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.080+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.080+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.080+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.080+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.080+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.081+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.081+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.081+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.081+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.083+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.083+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.083+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.083+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.083+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.083+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.087+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.087+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.088+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.088+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.088+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.088+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.091+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.091+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.091+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.091+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.091+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.091+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.092+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.092+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.092+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.092+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.093+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.093+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.093+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.093+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.097+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.097+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.097+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.097+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.097+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.097+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.106+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.106+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.106+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.106+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.106+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.106+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.109+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.109+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.109+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.109+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.109+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.109+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.110+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.110+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.110+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.110+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.111+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.111+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.111+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.111+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.111+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.111+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.114+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.114+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.114+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.114+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.115+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.115+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.116+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.116+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.116+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.117+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.117+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.117+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.117+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.117+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.163+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.163+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.164+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.164+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.168+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.168+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.174+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.174+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.174+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.174+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.193+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.193+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.193+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.193+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.193+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.193+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.226+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.226+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.226+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.226+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.265+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.265+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.265+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.265+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.295+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.295+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.296+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.296+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.296+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.296+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.301+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.301+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.301+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.301+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.302+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.302+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.302+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.302+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.302+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.308+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.308+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.313+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.313+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.314+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.314+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.317+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.317+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.317+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.317+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.318+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.318+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.318+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.318+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.318+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.318+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.323+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.364+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.364+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.364+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.364+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.394+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.394+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.394+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.394+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.394+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.394+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.395+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.395+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.395+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.395+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.399+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.399+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.399+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.399+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.418+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.418+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.437+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.437+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.489+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.491+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.491+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.491+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.491+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.491+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.491+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.492+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.492+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.492+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.492+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.500+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.501+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.501+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.501+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.501+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.502+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.502+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.502+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.502+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.502+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.502+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.503+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.503+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.503+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.503+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.503+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.503+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.519+0000 7f98854ae640 -1 rbd::mirror::InstanceWatcher: 0x555f64b9e380 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4885 object: (2) No such file or directory 2024-04-23T21:59:04.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.520+0000 7f98854ae640 -1 rbd::mirror::InstanceWatcher: 0x555f64b9e700 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4885 object: (2) No such file or directory 2024-04-23T21:59:04.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.520+0000 7f987eca1640 -1 rbd::mirror::InstanceWatcher: 0x555f64b9e380 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:59:04.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.520+0000 7f987eca1640 -1 rbd::mirror::InstanceWatcher: 0x555f64b9e700 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:59:04.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.520+0000 7f9878c95640 -1 rbd::mirror::MirrorStatusUpdater 0x555f64c04360 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:04.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.520+0000 7f9878c95640 -1 rbd::mirror::MirrorStatusUpdater 0x555f64e02240 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:04.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.520+0000 7f98854ae640 -1 rbd::mirror::NamespaceReplayer: 0x555f64c02000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:59:04.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.520+0000 7f98854ae640 -1 rbd::mirror::NamespaceReplayer: 0x555f64c021a0 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:59:04.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.520+0000 7f987f4a2640 -1 rbd::mirror::MirrorStatusUpdater 0x555f64c04000 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:04.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.520+0000 7f987f4a2640 -1 rbd::mirror::MirrorStatusUpdater 0x555f64c04120 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:04.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.520+0000 7f9885caf640 -1 rbd::mirror::NamespaceReplayer: 0x555f64c02000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:59:04.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.520+0000 7f9885caf640 -1 rbd::mirror::NamespaceReplayer: 0x555f64c021a0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:59:04.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.523+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.523+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.526+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.526+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.541+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.541+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.541+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.541+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.541+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.541+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.572+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.572+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.572+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.572+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.572+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.572+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.598+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.598+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.598+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.598+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.602+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.602+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.602+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.602+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.611+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.611+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.611+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.611+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.615+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.615+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.615+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.615+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.615+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.615+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.618+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.618+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.618+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.618+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.619+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.619+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.619+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.619+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.619+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.619+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.623+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.623+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.623+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.623+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.624+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.625+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.625+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.625+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.625+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.625+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.625+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.626+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.626+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.626+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.626+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.626+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.626+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.627+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.627+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.627+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.627+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.628+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.628+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.628+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.628+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.628+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.628+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.630+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.630+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.630+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.630+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.640+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.640+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.640+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.640+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.640+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.640+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.642+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.642+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.642+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.642+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.642+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.642+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.648+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.648+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.648+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.648+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.650+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.650+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.650+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.650+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.650+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.650+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.659+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.686+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.686+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.689+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.689+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.708+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.708+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.708+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.708+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.721+0000 7f986b47a640 -1 rbd::mirror::RemotePoolPoller: 0x555f64f16000 handle_mirror_peer_ping: failed to ping remote mirror peer: (2) No such file or directory 2024-04-23T21:59:04.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.721+0000 7f986ac79640 -1 rbd::mirror::RemotePoolPoller: 0x555f64f16000 handle_mirror_peer_list: failed to retrieve mirror peers: (2) No such file or directory 2024-04-23T21:59:04.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.721+0000 7f986ac79640 -1 rbd::mirror::PoolReplayer: 0x555f64aecf00 handle_remote_pool_meta_updated: remote pool metadata updated unexpectedly 2024-04-23T21:59:04.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.732+0000 7f9885caf640 -1 rbd::mirror::LeaderWatcher: 0x555f64aecc00 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (2) No such file or directory 2024-04-23T21:59:04.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.732+0000 7f9885caf640 -1 rbd::mirror::InstanceWatcher: 0x555f64b9e000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4885 object: (2) No such file or directory 2024-04-23T21:59:04.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.732+0000 7f987eca1640 -1 rbd::mirror::InstanceWatcher: 0x555f64b9e000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:59:04.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.733+0000 7f9878494640 -1 rbd::mirror::MirrorStatusUpdater 0x555f649685a0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:04.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.733+0000 7f98854ae640 -1 rbd::mirror::NamespaceReplayer: 0x555f65406000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:59:04.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.733+0000 7f987f4a2640 -1 rbd::mirror::MirrorStatusUpdater 0x555f64923560 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:04.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.733+0000 7f9885caf640 -1 rbd::mirror::NamespaceReplayer: 0x555f65406000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:59:04.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.736+0000 7f98854ae640 -1 rbd::mirror::LeaderWatcher: 0x555f64d8b200 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (2) No such file or directory 2024-04-23T21:59:04.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.736+0000 7f98854ae640 -1 rbd::mirror::InstanceWatcher: 0x555f65052000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4887 object: (2) No such file or directory 2024-04-23T21:59:04.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.737+0000 7f9871c87640 -1 rbd::mirror::InstanceWatcher: 0x555f65052000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:59:04.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.737+0000 7f986b47a640 -1 rbd::mirror::MirrorStatusUpdater 0x555f64e02b40 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:04.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.737+0000 7f98854ae640 -1 rbd::mirror::NamespaceReplayer: 0x555f64c024e0 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:59:04.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.737+0000 7f9871486640 -1 rbd::mirror::MirrorStatusUpdater 0x555f64dd3440 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:04.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:04.737+0000 7f9885caf640 -1 rbd::mirror::NamespaceReplayer: 0x555f64c024e0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:59:04.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.738+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.738+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.738+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.738+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.770+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.770+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.770+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.770+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.772+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.772+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.772+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.772+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.772+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.772+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.773+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.775+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.775+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.775+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.775+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.778+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.778+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.778+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.778+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.780+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.780+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.786+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.786+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.786+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.786+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.786+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.786+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.789+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.789+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.789+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.789+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.801+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.801+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.801+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.801+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.806+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.806+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.806+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.806+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.806+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.806+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.819+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.820+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.825+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.825+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.826+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.826+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.826+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.826+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.826+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.826+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.832+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.832+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.832+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.832+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.834+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.834+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.834+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.834+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.836+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.836+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.836+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.836+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.836+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.836+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.842+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.842+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.842+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.842+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.855+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.855+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.855+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.855+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.856+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.856+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.866+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.866+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.866+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.866+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.886+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.886+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.887+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.887+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.887+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.887+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.888+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.888+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.888+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.888+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.889+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.889+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.899+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.899+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.899+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.899+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.920+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.920+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.920+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.920+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.920+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.924+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.924+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.924+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.924+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.927+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.927+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.927+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.927+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.927+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.928+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.928+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.928+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.935+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.936+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.936+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.938+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.938+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.938+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.938+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.941+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.941+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.944+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.944+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.948+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.948+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.948+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.948+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.948+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.949+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.950+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.950+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.950+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.950+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.950+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.950+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.951+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.951+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.951+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.951+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.951+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.951+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.952+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.952+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.952+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.952+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.953+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.953+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.953+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.972+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.984+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.984+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.984+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.984+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:04.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:04.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:04.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.032+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.032+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.032+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.032+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.033+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.038+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.038+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.038+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.038+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.040+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.040+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.040+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.040+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.040+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.040+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.041+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.041+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.041+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.041+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.043+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.043+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.043+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.043+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.044+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.044+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.044+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.044+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.044+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.044+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.045+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.045+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.045+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.045+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.045+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.045+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.046+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.056+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.056+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.056+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.056+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.056+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.056+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.057+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.086+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.087+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.087+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.087+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.087+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.092+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.092+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.092+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.092+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.093+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.093+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.093+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.093+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.118+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.118+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.122+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.122+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.122+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.122+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.122+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.122+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.125+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.127+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.128+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.129+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.129+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.129+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.129+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.129+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.129+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.132+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.133+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.133+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.133+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.133+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.133+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.133+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.134+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.134+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.134+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.134+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.135+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.135+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.135+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.135+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.135+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.135+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.136+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.136+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.136+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.136+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.137+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.137+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.137+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.137+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.137+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.137+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.139+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.139+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.139+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.139+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.139+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.139+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.140+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.140+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.140+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.140+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.140+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.140+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.141+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.141+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.145+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.145+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.145+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.145+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.147+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.147+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.147+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.147+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.147+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.147+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.152+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.152+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.152+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.152+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.153+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.153+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.153+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.153+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.181+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.181+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.185+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.185+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.185+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.185+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.185+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.185+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.186+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.186+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.186+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.186+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.205+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.205+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.208+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.208+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.208+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.208+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.209+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.209+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.209+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.209+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.218+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.218+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.218+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.218+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.220+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.220+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.220+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.220+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.221+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.221+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.221+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.221+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.222+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.222+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.222+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.222+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.234+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.234+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.234+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.234+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.236+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.236+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.236+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.236+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.236+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.236+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.238+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.238+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.238+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.238+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.241+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.241+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.241+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.241+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.242+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.242+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.242+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.242+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.242+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.242+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.243+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.243+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.243+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.243+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.243+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.243+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.244+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.244+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.244+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.244+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.246+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.246+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.246+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.246+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.246+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.246+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.256+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.256+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.256+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.256+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.269+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.269+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.275+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.275+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.275+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.275+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.283+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.283+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.283+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.283+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.286+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.286+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.286+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.286+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.297+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.297+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.297+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.297+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.304+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.304+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.304+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.304+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.335+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.335+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.386+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.386+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.386+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.386+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.389+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.389+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.389+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.389+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.399+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.399+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.407+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.407+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.407+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.407+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.425+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.425+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.462+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.462+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.464+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.464+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.464+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.464+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.503+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.503+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.513+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.513+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.513+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.513+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.513+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.513+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.539+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.584+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.584+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.584+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.584+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.585+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.585+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.585+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.585+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.585+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.585+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.587+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.587+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.587+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.587+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.608+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.608+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.608+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.608+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.622+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.629+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.686+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.686+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.686+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.686+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.686+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.686+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.688+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.688+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.700+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.716+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.716+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.717+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.718+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.718+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.718+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.718+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.719+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.719+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.719+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.719+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.719+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.719+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.721+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.721+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.721+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.721+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.721+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.721+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.728+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.728+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.728+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.728+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.729+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.729+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.729+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.729+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.729+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.729+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.735+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.735+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.735+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.735+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.735+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.735+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.736+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.736+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.736+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.736+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.738+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.738+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.738+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.738+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.751+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.751+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.751+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.751+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.754+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.754+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.754+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.754+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.758+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.758+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.758+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.758+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.758+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.758+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.811+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.811+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.811+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.811+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.817+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.818+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.850+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.850+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.850+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.850+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.856+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.856+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.858+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.858+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.866+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.866+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.866+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.866+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.869+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.891+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.894+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.896+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.896+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.952+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.952+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.952+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.952+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.954+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.954+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.955+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.955+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.955+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.955+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.967+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.967+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.967+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.967+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.967+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.967+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.968+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.968+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.968+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.968+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.969+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.990+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.990+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.990+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.990+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.993+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.993+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.993+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.993+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.998+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.998+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.998+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.998+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.999+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.999+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:05.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.999+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:05.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.999+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.999+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:05.999+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.000+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.000+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.000+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.000+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.000+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.000+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.014+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.078+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.078+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.082+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.083+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.083+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.083+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.083+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.084+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.089+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.106+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.106+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.111+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.111+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.118+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.118+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.130+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.131+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.154+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.154+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.154+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.154+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.154+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.154+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.155+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.168+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.168+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.168+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.168+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.172+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.172+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.172+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.172+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.172+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.172+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.174+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.174+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.174+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.174+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.174+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.174+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.175+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.181+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.181+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.181+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.181+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.185+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.185+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.192+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.192+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.193+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.193+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.193+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.193+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.193+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.193+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.198+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.198+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.198+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.198+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.199+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.200+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.201+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.202+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.220+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.220+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.222+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.222+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.267+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.268+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.269+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.269+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.269+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.269+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.275+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.275+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.283+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.283+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.287+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.287+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.288+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.288+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.288+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.288+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.291+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.291+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.292+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.292+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.293+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.293+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.293+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.293+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.295+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.295+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.308+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.308+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.316+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.316+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.316+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.316+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.317+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.317+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.318+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.318+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.319+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.334+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.337+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.337+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.337+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.337+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.342+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.344+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.344+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.344+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.344+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.345+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.345+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.345+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.345+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.348+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.348+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.348+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.348+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.351+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.352+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.353+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.354+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.355+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.356+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.364+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.364+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.364+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.364+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.371+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.372+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.375+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.376+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.382+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.386+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.386+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.386+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.386+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.389+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.389+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.395+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.395+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.395+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.395+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.399+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.399+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.399+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.399+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.408+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.408+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.415+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.415+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.420+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.420+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.420+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.420+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.420+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.420+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.437+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.437+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.439+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.439+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.439+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.439+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.447+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.447+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.447+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.447+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.448+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.448+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.451+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.451+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.451+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.451+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.462+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.462+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.462+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.462+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.504+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.514+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.530+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.530+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.531+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.531+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.531+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.531+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.531+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.531+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.567+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.567+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.569+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.569+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.569+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.569+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.569+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.569+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.572+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.572+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.572+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.573+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.578+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.587+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.587+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.598+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.598+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.598+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.598+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.599+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.602+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.602+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.602+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.602+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.611+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.611+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.618+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.618+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.618+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.618+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.618+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.618+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.619+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.619+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.619+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.619+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.620+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.621+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.623+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.623+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.632+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.633+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.634+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.638+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.639+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.640+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.640+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.640+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.640+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.641+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.642+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.642+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.642+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.642+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.644+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.645+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.646+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.647+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.648+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.648+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.648+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.648+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.649+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.650+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.650+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.650+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.650+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.657+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.665+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.666+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.666+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.666+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.666+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.666+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.666+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.667+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.668+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.686+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.686+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.687+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.689+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.689+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.689+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.689+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.701+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.702+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.703+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.703+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.704+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.704+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.704+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.704+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.704+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.704+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.705+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.705+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.705+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.705+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.705+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.705+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.706+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.706+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.706+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.706+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.706+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.706+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.707+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.720+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.721+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.731+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.733+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.734+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.734+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.734+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.734+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.756+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.756+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.757+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.758+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.759+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.760+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.761+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.762+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.763+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.764+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.766+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.767+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.768+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.789+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.789+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.789+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.789+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.789+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.789+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.790+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.790+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.790+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.790+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.791+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.791+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.791+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.791+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.791+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.791+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.793+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.793+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.793+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.793+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.809+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.810+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.811+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.811+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.811+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.811+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.811+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.814+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.815+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.816+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.824+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.824+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.824+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.824+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.824+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.824+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.825+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.825+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.825+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.825+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.825+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.825+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.841+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.841+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.854+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.854+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.864+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.865+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.867+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.890+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.892+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.893+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.895+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.896+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.896+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.896+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.896+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.897+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.898+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.899+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.900+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.901+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.902+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.903+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.904+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.905+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.906+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.910+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.913+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.914+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.915+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.916+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.919+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.920+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.920+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.920+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.920+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.920+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.921+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.922+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.923+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.924+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.924+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.924+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.924+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.925+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.926+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.926+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.926+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.926+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.927+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.927+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.927+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.927+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.927+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.927+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.928+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.928+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.928+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.928+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.928+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.928+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.933+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.935+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.935+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.935+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.935+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.936+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.940+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.941+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.942+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.943+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.944+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.945+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.965+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.966+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.980+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.981+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.984+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.984+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:06.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:06.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:06.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.016+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.016+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.021+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.026+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.029+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.034+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.046+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.047+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.047+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.047+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.047+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.048+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.048+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.048+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.048+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.049+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.050+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.050+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.050+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.050+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.050+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.050+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.051+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.051+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.051+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.051+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.052+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.053+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.054+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.054+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.054+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.054+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.056+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.056+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.056+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.056+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.056+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.056+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.057+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.057+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.057+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.057+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.057+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.057+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.058+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.058+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.058+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.058+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.060+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.060+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.060+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.060+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.060+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.060+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.063+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.063+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.063+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.063+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.066+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.068+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.072+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.072+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.090+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.090+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.090+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.094+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.095+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.096+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.098+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.100+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.101+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.102+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.103+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.104+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.105+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.106+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.106+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.106+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.106+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.107+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.108+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.109+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.109+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.109+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.109+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.109+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.109+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.110+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.110+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.110+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.110+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.111+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.111+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.111+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.111+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.112+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.113+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.114+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.114+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.114+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.114+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.115+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.115+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.117+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.117+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.117+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.118+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.119+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.120+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.122+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.122+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.122+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.122+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.138+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.139+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.139+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.139+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.139+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.139+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.139+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.140+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.140+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.140+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.140+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.140+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.140+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.141+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.141+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.141+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.141+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.152+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.152+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.155+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.155+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.155+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.155+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.158+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.158+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.159+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.159+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.159+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.159+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.171+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.172+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.172+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.172+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.173+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.176+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.179+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.179+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.179+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.179+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.179+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.179+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.203+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.208+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.208+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.208+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.208+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.222+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.223+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.224+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.225+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.234+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.234+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.234+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.234+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.234+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.235+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.236+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.236+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.236+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.236+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.237+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.238+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.238+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.238+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.238+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.238+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.238+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.239+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.240+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.241+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.241+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.241+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.241+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.242+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.242+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.242+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.242+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.242+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.242+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.243+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.243+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.243+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.243+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.244+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.244+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.244+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.244+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.244+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.244+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.245+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.246+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.246+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.246+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.246+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.247+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.248+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.249+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.250+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.251+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.252+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.255+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.255+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.255+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.255+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.255+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.255+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.256+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.256+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.256+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.256+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.257+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.258+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.265+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.269+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.270+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.271+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.272+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.273+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.276+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.281+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.289+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.297+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.297+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.304+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.304+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.325+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.326+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.327+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.335+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.335+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.343+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.357+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.358+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.359+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.360+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.361+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.366+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.368+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.369+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.383+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.384+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.391+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.394+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.394+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.394+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.394+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.394+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.394+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.395+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.395+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.395+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.395+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.396+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.411+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.411+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.411+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.411+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.412+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.412+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.412+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.412+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.412+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.412+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.413+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.413+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.413+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.413+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.413+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.413+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.418+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.418+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.418+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.418+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.418+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.418+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.424+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.425+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.425+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.425+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.425+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.426+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.428+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.430+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.441+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.442+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.445+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.445+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.445+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.445+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.445+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.445+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.446+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.446+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.446+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.446+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.446+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.446+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.447+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.447+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.447+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.447+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.448+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.448+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.448+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.448+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.448+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.448+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.451+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.451+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.451+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.451+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.453+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.454+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.454+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.454+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.454+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.454+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.454+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.455+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.460+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.460+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.460+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.460+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.460+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.460+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.462+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.462+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.462+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.462+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.463+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.464+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.464+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.464+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.464+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.464+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.464+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.477+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.478+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.479+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.494+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.495+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.504+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.504+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.511+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.523+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.523+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.523+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.523+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.524+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.524+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.524+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.524+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.524+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.524+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.525+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.525+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.525+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.525+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.525+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.525+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.526+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.526+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.526+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.526+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.531+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.531+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.533+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.534+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.535+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.539+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.548+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.548+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.548+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.549+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.550+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.551+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.562+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.562+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.567+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.574+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.575+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.576+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.577+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.579+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.580+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.581+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.591+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.592+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.593+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.594+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.595+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.602+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.602+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.602+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.602+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.603+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.630+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.630+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.631+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.642+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.643+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.660+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.661+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.662+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.679+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.682+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.683+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.684+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.684+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.684+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.684+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.684+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.684+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.685+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.685+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.685+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.685+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.686+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.687+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.688+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.689+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.690+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.691+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.692+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.693+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.694+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.695+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.696+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.697+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.698+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.698+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.698+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.698+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.699+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.708+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.708+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.709+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.710+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.711+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.712+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.713+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.714+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.715+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.716+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.716+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.721+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.722+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.723+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.724+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.725+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.726+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.727+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.728+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.728+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.728+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.728+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.729+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.729+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.730+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.731+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.732+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.734+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.735+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.735+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.735+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.735+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.736+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.736+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.736+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.736+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.736+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.736+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.737+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.739+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.740+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.741+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.743+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.744+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.745+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.746+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.747+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.748+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.749+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.751+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.751+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.751+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.751+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.751+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.751+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.752+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.753+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.754+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.754+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.754+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.754+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.754+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.754+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.758+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.758+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.758+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.758+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.765+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.775+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.780+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.781+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.782+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.783+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.784+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.785+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.786+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.787+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.788+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.789+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.790+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.791+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.792+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.793+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.794+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.795+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.796+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.797+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.798+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.799+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.800+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.801+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.801+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.802+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.803+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.804+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.805+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.806+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.807+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.808+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.812+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.813+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.816 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.821+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.822+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.823+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.824+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.824+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.824+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.824+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.825+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.825+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.825+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.825+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.825+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.826+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.827+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.828+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.829+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.830+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.831+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.833+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.834+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.834+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.834+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.834+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.835+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.836+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.836+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.836+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.836+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.836+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.836+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.837+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.838+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.839+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.840+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.841+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.842+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.843+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.844+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.845+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.846+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.847+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.848+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.849+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.850+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.850+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.850+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.850+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.850+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.850+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.851+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.852+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.853+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.854+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.855+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.856+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.857+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.858+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.859+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.860+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.861+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.862+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.863+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.868+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.870+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.871+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.872+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.873+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.874+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.875+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.876+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.877+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.878+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.879+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.880 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.880+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.881+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.882+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.883+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.884+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.885+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.885+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.885+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.885+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.885+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.885+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.886+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.886+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.886+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.886+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.887+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.887+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.887+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.887+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.887+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.887+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.888+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.888+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.888+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.888+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.888+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.888+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.889+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.889+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.889+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.889+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.894+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.894+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.894+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.894+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.894+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.894+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.899+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.899+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.899+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.899+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.899+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.899+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.907+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.908+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.909+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.911+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.912+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.917+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.918+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.929+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.930+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.931+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.932+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.934+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.935+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.936+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.936+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.936+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.936+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.937+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.938+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.939+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.940+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.940+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.940+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.940+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.940+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.940+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.941+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.941+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.941+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.941+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.941+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.941+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.944+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.944+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.944+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.944+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.944+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.944+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.947+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.956+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.957+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.958+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.959+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.960+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.961+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.962+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.963+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.964+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.968+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.970+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.973+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.974+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.975+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.975+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.976+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.977+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.978+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.982+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.983+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.985+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.986+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.986+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.986+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.986+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.986+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.986+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.987+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.987+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.987+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.987+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.988+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.989+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.991+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.992+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.993+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.993+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.993+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.994+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.995+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.996+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.997+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:07.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:07.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:07.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.000+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.000+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.001+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.002+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.003+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.004+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.005+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.006+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.007+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.008+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.009+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.010+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.011+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.012+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.013+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.022+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.023+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.024+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.024+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.035+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.036+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.037+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.037+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.037+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.037+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.038+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.038+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.038+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.038 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.038+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.039+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.039+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.039+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.039+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.040+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.040+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.040+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.040+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.041+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.041+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.041+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.041+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.041+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.041+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.042+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.043+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.043+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.043+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.043+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.043+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.043+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.044+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.044+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.044+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.044+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.045+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.047+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.047+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.058+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.058+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.058+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.058+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.059+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.060+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.060+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.060+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.060+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.061+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.062+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.063+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.063+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.063+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.063+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.064+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.065+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.070+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.070+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.070+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.070+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.071+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.073+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.074+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.074+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.074+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.074+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.075+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.075+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.075+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.075+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.075+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.075+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.090+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.090+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.090+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.090+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.098+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.118+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.118+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.118+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.118+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.123+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.124+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.125+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.126+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.143+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.144+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.145+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.146+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.147+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.148+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.149+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.150+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.151+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.152+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.153+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.154+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.155+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.157+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.158+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.159+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.160+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.161+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.162+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.163+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.164+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.165+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.166+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.167+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.168+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.168+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.168+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.168+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.169+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.178+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.178+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.178+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.178+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.178+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.178+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.179+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.180+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.181+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.181+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.181+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.181+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.182+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.182+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.182+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.182+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.182+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.183+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.184+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.185+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.186+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.187+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.188+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.189+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.190+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.191+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.194+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.195+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.196+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.197+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.199+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.203+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.204+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.205+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.205+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.205+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.205+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.206+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.207+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.211+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.212+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.213+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.214+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.215+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.216+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.217+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.218+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.219+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.222+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.226+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.226+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.226+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.226+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.227+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.228+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.229+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.230+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.231+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.232+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.233+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.253+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.254+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.255+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.255+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.256+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.256+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.260+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.261+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.262+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.263+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.264+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.265+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.265+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.265+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.265+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.265+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.265+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.266+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.274+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.275+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.277+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.278+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.279+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.280+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.283+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.283+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.284+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.285+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.286+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.286+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.286+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.286+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.286+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.286+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.290+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.291+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.291+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.291+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.291+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.292+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.292+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.292+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.292+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.296+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.297+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.297+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.297+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.298+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.299+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.300+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.301+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.301+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.301+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.301+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.301+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.301+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.302+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.302+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.302+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.302+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.303+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.304+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.304+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.304+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.304+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.304+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.304+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.305+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.306+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.307+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.309+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.310+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.320+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.321+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.328+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.329+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.330+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.331+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.332+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.333+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.335+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.335+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.335+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.335+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.335+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.335+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.336+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.337+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.338+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.339+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.340+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.341+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.345+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.345+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.345+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.345+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.346+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.347+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.348+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.348+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.348+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.348+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.349+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.350+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.362+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.363+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.364+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.364+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.364+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.364+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.367+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.370+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.373+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.374+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.377+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.378+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.379+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.380+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.381+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.385+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.387+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.388+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.389+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.389+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.392+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.393+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.397+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.398+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.399+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.399+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.399+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.399+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.399+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.399+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.400+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.401+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.402+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.403+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.404+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.405+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.406+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.409+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.410+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.414+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.415+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.416+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.417+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.419+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.419+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.421+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.422+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.423+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.423+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.423+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.423+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.425+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.425+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.427+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.429+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.431+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.432+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.433+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.440+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.443+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.444+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.449+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.450+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.452+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.456+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.457+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.458+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.460+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.460+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.460+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.462+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.462+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.464+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.464+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.464+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.464+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.465+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.466+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.468+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.469+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.470+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.471+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.472+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.473+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.474+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.475+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.476+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.481+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.482+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.483+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.484+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.486+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.487+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.488+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.490+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.493+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.496+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.498+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.500+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.504+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.504+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.505+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.506+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.507+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.508+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.509+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.510+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.512+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.515+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.516+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.517+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.518+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.519+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.520+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.521+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.522+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.523+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.523+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.523+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.523+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.524+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.524+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.524+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.524+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.525+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.525+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.525+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.525+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.525+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.525+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.526+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.526+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.526+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.526+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.527+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.528+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.529+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.530+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.530+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.530+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.530+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.531+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.531+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.531+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.531+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.532+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.536+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.537+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.538+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.540+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.541+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.541+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.542+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.543+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.544+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.545+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.546+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.547+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.548+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.548+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.548+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.548+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.548+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.548+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.552+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.553+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.554+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.555+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.555+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.555+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.555+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.555+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.555+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.556+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.557 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.557+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.558+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.559+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.560+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.561+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.562+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.562+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.562+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.562+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.563+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.564+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.565+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.566+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.567+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.567+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.567+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.567+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.568+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.569+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.570+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.572+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.582+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.583+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.584+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.585+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.586+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.587+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.588+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.589+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.590+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.596+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.597+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.600+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.601+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.604+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.605+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.606+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.607 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.607+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.608+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.609+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.610+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.611+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.611+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.611+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.611+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.611+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.611+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.612+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.613 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.613+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.614+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.615+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.615+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.615+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.615+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.616+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.617+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.622 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.630 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.636+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.651+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.652+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.653+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.655+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.664+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.669+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.671+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.672+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.673+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.674+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.676+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.677+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.678+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.679+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.680+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.682+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.683+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.685 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.684+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.685+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.686+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.686+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.689+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.689+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.689+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.689+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.690+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.691+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.692+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.693+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.694+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.695+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.696+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.697+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.698+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.699+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.700+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.701+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.702+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.703+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.704+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.705+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.706+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.707+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.708+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.709+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.710+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.711 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.711+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.712+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.713+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.714+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.715+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.716+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.717 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.717+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.718 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.718+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.719+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.720+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.721+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.722+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.723+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.724+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.725+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.726 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.726+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.727+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.728+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.729+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.730 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.731 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.730+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.732+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.733+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.734+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.734+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.734+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.734+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.735+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.736+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.737+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.738+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.739+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.740+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.741+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.742+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.742+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.743+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.744+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.745+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.746+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.747+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.748+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.749+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.750+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.750+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.751+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.752+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.753+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.754+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.755+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.755+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.756 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.756+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.757+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.758+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.758+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.758+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.758+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.758+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.758+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.759 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.759+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.760+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.761 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.761+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.762+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.763+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.764+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.765+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.766+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.767+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.768+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.769+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.770+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.771+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.772+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.773+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.774+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.774+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.775+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.775+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.775+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.775+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.776+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.776+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.777 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.777+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.778+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.779+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.779+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.780+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.780+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.780+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.780+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.781+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.782+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.782+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.782+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.782+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.783+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.784+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.785+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.786+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.786+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.786+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.786+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.787+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.788+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.789+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.789+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.789+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.789+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.789+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.789+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.790+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.790+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.790+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.790+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.790+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.790+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.791+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.791+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.791+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.791+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.792+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.793+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.793+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.793+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.793+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.793+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.793+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.794+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.795+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.795+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.795+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.795+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.796+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.796+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.796+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.796+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.797+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.797+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.797+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.797+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.797+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.797+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.798+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.799+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.800+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.801+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.803 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.802+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.803+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.805 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.804+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.805+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.806+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.806+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.806+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.806+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.808 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.807+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.808+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.808+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.808+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.808+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.809+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.811 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.810+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.811+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.812+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.812+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.812+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.812+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.812+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.812+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.814 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.813+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.814+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.815+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.816+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.818 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.817+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.818+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.819+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.820+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.822 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.821+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.822+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.823+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.824+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.825 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.825+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.826+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.827+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.828+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.830 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.829+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.830+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.831+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.832+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.833+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.834 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.834+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.835+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.837 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.836+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.837+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.838+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.839+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.840+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.841+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.841+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.841+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.841+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.841+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.841+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.842+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.842+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.842+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.842+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.843+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.843+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.843+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.843+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.843+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.843+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.844+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.844+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.844+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.844+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.844+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.844+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.845+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.846+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.847+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.849 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.848+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.849+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.850+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.851+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.852 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.852+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.853 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.853+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.854+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.854+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.854+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.854 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.854+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.855+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.855+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.855 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.855+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.855+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.855+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.855+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.856+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.856+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.856+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.856+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.858 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.857+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.858+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.858+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.858+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.858+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.859+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.859+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.859+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.859+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.860 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.860+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.861+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.862+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.863 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.863+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.864 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.864+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.865 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.865+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.866+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.867 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.867+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.868 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.868+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.869 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.869+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.870+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.870+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.870+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.870+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.871+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.872 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.872+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.874 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.873+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.874+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.875+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.876+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.877 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.877+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.878+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.879 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.879+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.880+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.881 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.882 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.881+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.883 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.882+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.883+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.884+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.885+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.886+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.887+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.888+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.889+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.890+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.891+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.892+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.893+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.894+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.894+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.895+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.896+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.897+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.898+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.899+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.899+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.899+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.899+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.900+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.901+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.902+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.903+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.904+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.905+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.906+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.907+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.908+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.909+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.910+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.911+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.912+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.913+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.914+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.915+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.916+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.917+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.918 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.918+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.919+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.920+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.921+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.922+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.923+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.924+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.925+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.926+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.927+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.928+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.929+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.930+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.931+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.932+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.933+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.934+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.935+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.935+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.936+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.936+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.936+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.936+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.937+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.938+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.938+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.938+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.938+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.939+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.940+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.940+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.940+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.940+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.941+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.941+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.941+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.941+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.942+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.943+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.944+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.944+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.944+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.944+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.944+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.944+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.945+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.946+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.946+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.947+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.948 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.948+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.949+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.950+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.951+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.952+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.953+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.954+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.955+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.956+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.957+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.958 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.958+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.959+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.960+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.961+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.962+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.963 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.963+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.964+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.965+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.966 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.966+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.967+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.969+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.970+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.971+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.972+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.973+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.974 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.974+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.975 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.975+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.976+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.977+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.978+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.979 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.979+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.979+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.979+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.980+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.981+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.982+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.982+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.982+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.982+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.983+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.983+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.983+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.983+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.984+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.984+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.984+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.985+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.986+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.987 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.987+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.988+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.989+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.990 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.990+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.991 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.991+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.992 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.992+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.993 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.993+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.994+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.995 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.995+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.996+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.997+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.998 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.998+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:08.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:08.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:08.999+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.000+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.001+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.002+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.003+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.004 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.004+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.005+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.006 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.006+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.007+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.008+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.009+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.010+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.011+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.012 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.013 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.012+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.013+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.014+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.015 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.015+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.016+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.017 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.017+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.018+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.019+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.020 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.020+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.021+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.022+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.024 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.023+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.026 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.025+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.026+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.027+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.028+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.029+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.030+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.031 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.031+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.032+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.033+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.035 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.034+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.035+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.037 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.036+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.037+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.038+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.040 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.039+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.040+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.042 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.041+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.042+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.043+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.044+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.046 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.045+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.046+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.047+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.048+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.049 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.049+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.050+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.051+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.052+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.053+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.054 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.054+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.055+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.055+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.056+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.057+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.058+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.060 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.059+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.060+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.061+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.062+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.063 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.063+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.064+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.065+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.066+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.067+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.068+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.069 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.069+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.070+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.071+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.072+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.073+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.074+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.075+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.076+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.077+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.078+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.079+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.080+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.081+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.082+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.083+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.084+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.085+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.085+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.086+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.087+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.088+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.089+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.090+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.090+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.090+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.090+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.090+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.090+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.091+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.092+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.094 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.093+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.096 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.094+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.095+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.096+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.097+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.099 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.099+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.099+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.100+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.101+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.102+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.103+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.104+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.105+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.106 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.106+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.107+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.108+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.109 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.109+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.110+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.111+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.112 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.112+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.113+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.114+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.115+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.116 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.116+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.117+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.118+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.118+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.118+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.118+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.119+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.120+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.121+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.121+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.122+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.122+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.122+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.122+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.123+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.124 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.124+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.126+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.127+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.128+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.129 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.129+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.130+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.131+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.132+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.133+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.134 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.134+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.135+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.136+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.137+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.138+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.139+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.140 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.140+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.141+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.142 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.142+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.143+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.144+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.145+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.145+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.145+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.145+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.146+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.147+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.147+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.147+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.147+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.148+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.149+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.150 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.150+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.151+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.152+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.152+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.152+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.152+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.152+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.152+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.153+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.153+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.153+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.153+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.153+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.153+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.155+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.155+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.156+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.156+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.156+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.156+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.156+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.157+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.158+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.158+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.158+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.158+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.159+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.159+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.159+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.159+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.159+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.159+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.160 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.160+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.161+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.162+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.163+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.163+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.163+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.163+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.163+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.163+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.164+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.164+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.164+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.164+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.165+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.166 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.166+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.167+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.168+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.169 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.169+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.170+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.170+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.171 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.171+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.172+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.173+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.174+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.175+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.176+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.177+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.177+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.178+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.179+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.179+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.179+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.179+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.180+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.181+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.182+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.183+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.184 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.184+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.185+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.185+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.185+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.185+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.186+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.186+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.186+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.186+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.186+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.186+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.187 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.187+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.188+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.189+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.190+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.191+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.192+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.192+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.192+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.193+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.194+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.195+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.196+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.197+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.198+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.200+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.201+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.202+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.204+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.205+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.206 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.206+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.207+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.208+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.208+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.208+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.208+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.208+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.209+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.210+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.210+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.210+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.210+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.210+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.211+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.211+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.211+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.211+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.212+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.213+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.214+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.215+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.216+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.217+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.218+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.218+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.218+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.218+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.218+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.218+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.219+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.220+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.221+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.221+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.221+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.222+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.222+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.222+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.222+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.223+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.224+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.225+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.226+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.226+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.226+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.227+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.228+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.229+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.230+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.231+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.232+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.233+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.234+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.235 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.234+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.234+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.235+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.236+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.237+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.238+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.239+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.240+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.241 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.241+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.242+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.243+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.244 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.244+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.245+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.246+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.247+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.248+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.249+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.250+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.251+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.252+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.253+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.254+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.255+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.256 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.256+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.257+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.258+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.259+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.259+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.260+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.261+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.262+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.263+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.264+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.265+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.266+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.267+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.268+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.269 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.269+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.270+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.271+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.272+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.273+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.274+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.275+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.275+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.275+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.275+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.277 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.276+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.277+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.278+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.279+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.280+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.281+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.282+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.282+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.283+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.284+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.285+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.286+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.287+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.288+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.289+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.290+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.291+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.292+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.293+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.294+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.295+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.295+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.299 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.298+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.299+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.300+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.301+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.302+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.303+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.304+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.305+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.306+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.307+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.308+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.309+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.310+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.311+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.312+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.313+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.314+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.315+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.316+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.317+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.319 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.318+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.319+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.320+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.321+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.322+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.322+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.323+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.324+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.324+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.325+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.326+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.327+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.328+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.329+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.330+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.331+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.332+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.333+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.334+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.335+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.336+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.337+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.337+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.337+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.337+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.338+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.339+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.340+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.341+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.342+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.343+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.344+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.345+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.346+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.347+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.348+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.349+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.350+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.351+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.352+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.353+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.354+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.355+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.356+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.357+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.358+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.359 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.359+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.360 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.360+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.361+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.362+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.363+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.364+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.365+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.365+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.366 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.366+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.367+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.368 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.368+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.369 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.369+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.370+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.371+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.372+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.373+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.374 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.374+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.375 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.375+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.376+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.377+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.378+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.379+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.380+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.381 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.381+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.382+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.383 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.383+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.384 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.384+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.385+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.386+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.387+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.388+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.389+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.390+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.390+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.391 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.391+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.392+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.393+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.394+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.395 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.395+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.396+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.397 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.397+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.398+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.399+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.400+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.401+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.402+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.403+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.404+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.405+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.406+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.407 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.407+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.408+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.409+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.410+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.411+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.412+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.413+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.414 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.414+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.415+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.415+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.415+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.415+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.416+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.417+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.418+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.419+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.420+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.421+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.422+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.423+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.425+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.426 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.426+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.427+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.428+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.429+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.430 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.430+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.431+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.432+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.433 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.433+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.434+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.435 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.435+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.436 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.436+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.437+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.438+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.439 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.439+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.440 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.440+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.441+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.442+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.443 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.443+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.444+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.445+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.446+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.447+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.448+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.449+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.450+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.451+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.452+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.453+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.454+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.455+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.456+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.457+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.458+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.459+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.459+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.460+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.460+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.461+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.461+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.462+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.463+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.464+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.465+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.467 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.466+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.467+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.467+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.468+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.470 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.469+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.470+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.471+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.472+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.473 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.473+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.475 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.474+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.475+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.476 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.476+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.477+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.478+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.479 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.479+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.480+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.480+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.481+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.482+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.483+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.484+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.485+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.485+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.486+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.487+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.488+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.489+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.490+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.491+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.492+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.493+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.494+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.495+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.496+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.497+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.498 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.497+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.498+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.499+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.499+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.501 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.501+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.502+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.503 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.503+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.504+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.504+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.504+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.505+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.506+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.507+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.508+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.509+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.510+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.511+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.512+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.513 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.513+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.514+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.515+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.516 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.516+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.517+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.518+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.519+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.521 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.520+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.522 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.521+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.522+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.523 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.523+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.524 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.524+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.525+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.526+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.527+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.528+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.529+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.530+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.531+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.532 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.532+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.533+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.534+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.535+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.536+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.537+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.538+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.540+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.541+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.542+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.543+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.544 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.544+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.545+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.546+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.547+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.548+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.548 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.548+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.548+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.548+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.549+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.550+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.551+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.552+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.553+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.554+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.555+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.556+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.557+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.558+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.559+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.561 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.560+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.561+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.562+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.562+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.562+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.563+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.564+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.565+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.566 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.566+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.568+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.569+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.569+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.569 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.570+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.571+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.571+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.573+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.574+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.575+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.576 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.576+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.577 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.577+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.578+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.579+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.580+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.581 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.581+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.582 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.582+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.583+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.584+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.584+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.584+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.584+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.584+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.584+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.585+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.585+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.585+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.585+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.586+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.587+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.587+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.587+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.587+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.588+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.589+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.590+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.591+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.592+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.593+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.594 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.594+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.595+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.596+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.597+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.598+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.599+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.600+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.601+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.602+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.605 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.603+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.604+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.605+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.606+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.607+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.608+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.608+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.608+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.608+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.609+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.610+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.611+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.612+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.613+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.614+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.615+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.616+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.617+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.618+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.619+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.620+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.621+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.622+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.623+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.624+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.625+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.626 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.626+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.627+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.628+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.629+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.630+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.632 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.631+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.632+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.633+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.634+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.635+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.635+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.636+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.637+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.637+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.638+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.640 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.639+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.640+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.641+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.642+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.643+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.644+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.645+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.646+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.647+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.648+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.649+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.650+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.651+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.652+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.653+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.654+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.654+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.655+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.656+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.656+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.657+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.658+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.658+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.659+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.660+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.661+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.662+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.663+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.663+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Stopped 2024-04-23T21:59:09.664 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2024-04-23T21:59:09.677 DEBUG:tasks.rbd_mirror.cluster1.client.mirror.2:waiting for process to exit 2024-04-23T21:59:09.677 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T21:59:09.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.664+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.665+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.666+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.667+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.668+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.669+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.670+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.670+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.671+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.672+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.673+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.674+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.675+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.675+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.676+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.677+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.678+0000 7fc7f96a6640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 106492) UID: 1000 2024-04-23T21:59:09.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.678+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.678+0000 7fc7e467c640 -1 rbd::mirror::InstanceWatcher: 0x5569a1884000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:59:09.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.680+0000 7fc7f8ea5640 -1 rbd::mirror::InstanceWatcher: 0x5569a1490000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5484 object: (2) No such file or directory 2024-04-23T21:59:09.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.680+0000 7fc7f8ea5640 -1 rbd::mirror::InstanceWatcher: 0x5569a1491c00 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5484 object: (2) No such file or directory 2024-04-23T21:59:09.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.680+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.680+0000 7fc7f1e97640 -1 rbd::mirror::InstanceWatcher: 0x5569a1490000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:59:09.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.680+0000 7fc7f2698640 -1 rbd::mirror::InstanceWatcher: 0x5569a1491c00 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:59:09.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory 2024-04-23T21:59:09.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.681+0000 7fc7ebe8b640 -1 rbd::mirror::MirrorStatusUpdater 0x5569a13d6360 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:09.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.681+0000 7fc7ebe8b640 -1 rbd::mirror::MirrorStatusUpdater 0x5569a13d65a0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:09.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.681+0000 7fc7f8ea5640 -1 rbd::mirror::NamespaceReplayer: 0x5569a13d2000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:59:09.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.681+0000 7fc7f8ea5640 -1 rbd::mirror::NamespaceReplayer: 0x5569a13d21a0 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:59:09.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.681+0000 7fc7f1e97640 -1 rbd::mirror::MirrorStatusUpdater 0x5569a13d6000 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:09.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.681+0000 7fc7f86a4640 -1 rbd::mirror::NamespaceReplayer: 0x5569a13d2000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:59:09.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.681+0000 7fc7f2698640 -1 rbd::mirror::MirrorStatusUpdater 0x5569a13d6120 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:09.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.681+0000 7fc7f8ea5640 -1 rbd::mirror::NamespaceReplayer: 0x5569a13d21a0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:59:09.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.681+0000 7fc7f86a4640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory 2024-04-23T21:59:09.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.681+0000 7fc7f8ea5640 -1 rbd::mirror::LeaderWatcher: 0x5569a12c8c00 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (2) No such file or directory 2024-04-23T21:59:09.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.682+0000 7fc7f86a4640 -1 rbd::mirror::InstanceWatcher: 0x5569a1af3500 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5484 object: (2) No such file or directory 2024-04-23T21:59:09.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.682+0000 7fc7f1e97640 -1 rbd::mirror::InstanceWatcher: 0x5569a1af3500 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2024-04-23T21:59:09.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.682+0000 7fc7eb68a640 -1 rbd::mirror::MirrorStatusUpdater 0x5569a10fa240 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:09.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.682+0000 7fc7f86a4640 -1 rbd::mirror::NamespaceReplayer: 0x5569a1b70000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2024-04-23T21:59:09.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.682+0000 7fc7f2698640 -1 rbd::mirror::MirrorStatusUpdater 0x5569a10858c0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2024-04-23T21:59:09.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi150.stderr:2024-04-23T21:59:09.682+0000 7fc7f8ea5640 -1 rbd::mirror::NamespaceReplayer: 0x5569a1b70000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (2) No such file or directory 2024-04-23T21:59:09.694 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Stopped 2024-04-23T21:59:09.694 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2024-04-23T21:59:09.711 DEBUG:tasks.rbd_mirror.cluster1.client.mirror.1:waiting for process to exit 2024-04-23T21:59:09.711 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T21:59:09.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi150.stderr:2024-04-23T21:59:09.735+0000 7f99a3eb7640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 60654) UID: 1000 2024-04-23T21:59:09.746 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Stopped 2024-04-23T21:59:09.746 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2024-04-23T21:59:09.757 DEBUG:tasks.rbd_mirror.cluster1.client.mirror.0:waiting for process to exit 2024-04-23T21:59:09.757 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T21:59:09.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi150.stderr:2024-04-23T21:59:09.787+0000 7f98864b0640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 83291) UID: 1000 2024-04-23T21:59:09.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Stopped 2024-04-23T21:59:09.798 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-23T21:59:09.809 INFO:tasks.ceph.ceph_manager.cluster2:waiting for clean 2024-04-23T21:59:09.809 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-04-23T21:59:10.132 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:59:10.132 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-23T21:59:10.144 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":1610,"stamp":"2024-04-23T21:59:09.102026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3107852,"kb_used_data":14248,"kb_used_omap":566,"kb_used_meta":3093001,"kb_avail":280007668,"statfs":{"total":289910292480,"available":286727852032,"internally_reserved":0,"allocated":14589952,"data_stored":7932410,"data_compressed":5808,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":579804,"internal_metadata":3167233828},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001652"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":812,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.508336+0000","last_change":"2024-04-23T21:19:03.617238+0000","last_active":"2024-04-23T21:58:57.508336+0000","last_peered":"2024-04-23T21:58:57.508336+0000","last_clean":"2024-04-23T21:58:57.508336+0000","last_became_active":"2024-04-23T21:19:03.616645+0000","last_became_peered":"2024-04-23T21:19:03.616645+0000","last_unstale":"2024-04-23T21:58:57.508336+0000","last_undegraded":"2024-04-23T21:58:57.508336+0000","last_fullsized":"2024-04-23T21:58:57.508336+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:25.085268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":812,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.507764+0000","last_change":"2024-04-23T21:19:03.617657+0000","last_active":"2024-04-23T21:58:57.507764+0000","last_peered":"2024-04-23T21:58:57.507764+0000","last_clean":"2024-04-23T21:58:57.507764+0000","last_became_active":"2024-04-23T21:19:03.616675+0000","last_became_peered":"2024-04-23T21:19:03.616675+0000","last_unstale":"2024-04-23T21:58:57.507764+0000","last_undegraded":"2024-04-23T21:58:57.507764+0000","last_fullsized":"2024-04-23T21:58:57.507764+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:11.044595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.508313+0000","last_change":"2024-04-23T21:19:03.617922+0000","last_active":"2024-04-23T21:58:57.508313+0000","last_peered":"2024-04-23T21:58:57.508313+0000","last_clean":"2024-04-23T21:58:57.508313+0000","last_became_active":"2024-04-23T21:19:03.616607+0000","last_became_peered":"2024-04-23T21:19:03.616607+0000","last_unstale":"2024-04-23T21:58:57.508313+0000","last_undegraded":"2024-04-23T21:58:57.508313+0000","last_fullsized":"2024-04-23T21:58:57.508313+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:36:08.192457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.504113+0000","last_change":"2024-04-23T21:19:03.616894+0000","last_active":"2024-04-23T21:58:57.504113+0000","last_peered":"2024-04-23T21:58:57.504113+0000","last_clean":"2024-04-23T21:58:57.504113+0000","last_became_active":"2024-04-23T21:19:03.616601+0000","last_became_peered":"2024-04-23T21:19:03.616601+0000","last_unstale":"2024-04-23T21:58:57.504113+0000","last_undegraded":"2024-04-23T21:58:57.504113+0000","last_fullsized":"2024-04-23T21:58:57.504113+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:46:19.804032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":812,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:56.506788+0000","last_change":"2024-04-23T21:19:03.615613+0000","last_active":"2024-04-23T21:58:56.506788+0000","last_peered":"2024-04-23T21:58:56.506788+0000","last_clean":"2024-04-23T21:58:56.506788+0000","last_became_active":"2024-04-23T21:19:03.615283+0000","last_became_peered":"2024-04-23T21:19:03.615283+0000","last_unstale":"2024-04-23T21:58:56.506788+0000","last_undegraded":"2024-04-23T21:58:56.506788+0000","last_fullsized":"2024-04-23T21:58:56.506788+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:14:30.533352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.981208+0000","last_change":"2024-04-23T21:19:03.617229+0000","last_active":"2024-04-23T21:58:57.981208+0000","last_peered":"2024-04-23T21:58:57.981208+0000","last_clean":"2024-04-23T21:58:57.981208+0000","last_became_active":"2024-04-23T21:19:03.616884+0000","last_became_peered":"2024-04-23T21:19:03.616884+0000","last_unstale":"2024-04-23T21:58:57.981208+0000","last_undegraded":"2024-04-23T21:58:57.981208+0000","last_fullsized":"2024-04-23T21:58:57.981208+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:11:38.295053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.981785+0000","last_change":"2024-04-23T21:19:03.617745+0000","last_active":"2024-04-23T21:58:57.981785+0000","last_peered":"2024-04-23T21:58:57.981785+0000","last_clean":"2024-04-23T21:58:57.981785+0000","last_became_active":"2024-04-23T21:19:03.617420+0000","last_became_peered":"2024-04-23T21:19:03.617420+0000","last_unstale":"2024-04-23T21:58:57.981785+0000","last_undegraded":"2024-04-23T21:58:57.981785+0000","last_fullsized":"2024-04-23T21:58:57.981785+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:10.514458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.504986+0000","last_change":"2024-04-23T21:19:03.616216+0000","last_active":"2024-04-23T21:58:57.504986+0000","last_peered":"2024-04-23T21:58:57.504986+0000","last_clean":"2024-04-23T21:58:57.504986+0000","last_became_active":"2024-04-23T21:19:03.615832+0000","last_became_peered":"2024-04-23T21:19:03.615832+0000","last_unstale":"2024-04-23T21:58:57.504986+0000","last_undegraded":"2024-04-23T21:58:57.504986+0000","last_fullsized":"2024-04-23T21:58:57.504986+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:25:34.897612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":947,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.507083+0000","last_change":"2024-04-23T21:18:58.602249+0000","last_active":"2024-04-23T21:58:57.507083+0000","last_peered":"2024-04-23T21:58:57.507083+0000","last_clean":"2024-04-23T21:58:57.507083+0000","last_became_active":"2024-04-23T21:18:58.600198+0000","last_became_peered":"2024-04-23T21:18:58.600198+0000","last_unstale":"2024-04-23T21:58:57.507083+0000","last_undegraded":"2024-04-23T21:58:57.507083+0000","last_fullsized":"2024-04-23T21:58:57.507083+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:57.592214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:57.592214+0000","last_clean_scrub_stamp":"2024-04-23T21:18:57.592214+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:36:39.822147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":5808,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738855,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1232236,"kb_used_data":4768,"kb_used_omap":141,"kb_used_meta":1227314,"kb_avail":93139604,"statfs":{"total":96636764160,"available":95374954496,"internally_reserved":0,"allocated":4882432,"data_stored":2840926,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":145248,"internal_metadata":1256769696},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738856,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":894708,"kb_used_data":4712,"kb_used_omap":180,"kb_used_meta":889803,"kb_avail":93477132,"statfs":{"total":96636764160,"available":95720583168,"internally_reserved":0,"allocated":4825088,"data_stored":2250558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":184673,"internal_metadata":911158943},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738855,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":980908,"kb_used_data":4768,"kb_used_omap":244,"kb_used_meta":975883,"kb_avail":93390932,"statfs":{"total":96636764160,"available":95632314368,"internally_reserved":0,"allocated":4882432,"data_stored":2840926,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":249883,"internal_metadata":999305189},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T21:59:10.145 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-04-23T21:59:10.628 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:59:10.628 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-23T21:59:10.640 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":1610,"stamp":"2024-04-23T21:59:09.102026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3107852,"kb_used_data":14248,"kb_used_omap":566,"kb_used_meta":3093001,"kb_avail":280007668,"statfs":{"total":289910292480,"available":286727852032,"internally_reserved":0,"allocated":14589952,"data_stored":7932410,"data_compressed":5808,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":579804,"internal_metadata":3167233828},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001652"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":812,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.508336+0000","last_change":"2024-04-23T21:19:03.617238+0000","last_active":"2024-04-23T21:58:57.508336+0000","last_peered":"2024-04-23T21:58:57.508336+0000","last_clean":"2024-04-23T21:58:57.508336+0000","last_became_active":"2024-04-23T21:19:03.616645+0000","last_became_peered":"2024-04-23T21:19:03.616645+0000","last_unstale":"2024-04-23T21:58:57.508336+0000","last_undegraded":"2024-04-23T21:58:57.508336+0000","last_fullsized":"2024-04-23T21:58:57.508336+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:25.085268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":812,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.507764+0000","last_change":"2024-04-23T21:19:03.617657+0000","last_active":"2024-04-23T21:58:57.507764+0000","last_peered":"2024-04-23T21:58:57.507764+0000","last_clean":"2024-04-23T21:58:57.507764+0000","last_became_active":"2024-04-23T21:19:03.616675+0000","last_became_peered":"2024-04-23T21:19:03.616675+0000","last_unstale":"2024-04-23T21:58:57.507764+0000","last_undegraded":"2024-04-23T21:58:57.507764+0000","last_fullsized":"2024-04-23T21:58:57.507764+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:11.044595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.508313+0000","last_change":"2024-04-23T21:19:03.617922+0000","last_active":"2024-04-23T21:58:57.508313+0000","last_peered":"2024-04-23T21:58:57.508313+0000","last_clean":"2024-04-23T21:58:57.508313+0000","last_became_active":"2024-04-23T21:19:03.616607+0000","last_became_peered":"2024-04-23T21:19:03.616607+0000","last_unstale":"2024-04-23T21:58:57.508313+0000","last_undegraded":"2024-04-23T21:58:57.508313+0000","last_fullsized":"2024-04-23T21:58:57.508313+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:36:08.192457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.504113+0000","last_change":"2024-04-23T21:19:03.616894+0000","last_active":"2024-04-23T21:58:57.504113+0000","last_peered":"2024-04-23T21:58:57.504113+0000","last_clean":"2024-04-23T21:58:57.504113+0000","last_became_active":"2024-04-23T21:19:03.616601+0000","last_became_peered":"2024-04-23T21:19:03.616601+0000","last_unstale":"2024-04-23T21:58:57.504113+0000","last_undegraded":"2024-04-23T21:58:57.504113+0000","last_fullsized":"2024-04-23T21:58:57.504113+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:46:19.804032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":812,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:56.506788+0000","last_change":"2024-04-23T21:19:03.615613+0000","last_active":"2024-04-23T21:58:56.506788+0000","last_peered":"2024-04-23T21:58:56.506788+0000","last_clean":"2024-04-23T21:58:56.506788+0000","last_became_active":"2024-04-23T21:19:03.615283+0000","last_became_peered":"2024-04-23T21:19:03.615283+0000","last_unstale":"2024-04-23T21:58:56.506788+0000","last_undegraded":"2024-04-23T21:58:56.506788+0000","last_fullsized":"2024-04-23T21:58:56.506788+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:14:30.533352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.981208+0000","last_change":"2024-04-23T21:19:03.617229+0000","last_active":"2024-04-23T21:58:57.981208+0000","last_peered":"2024-04-23T21:58:57.981208+0000","last_clean":"2024-04-23T21:58:57.981208+0000","last_became_active":"2024-04-23T21:19:03.616884+0000","last_became_peered":"2024-04-23T21:19:03.616884+0000","last_unstale":"2024-04-23T21:58:57.981208+0000","last_undegraded":"2024-04-23T21:58:57.981208+0000","last_fullsized":"2024-04-23T21:58:57.981208+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:11:38.295053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.981785+0000","last_change":"2024-04-23T21:19:03.617745+0000","last_active":"2024-04-23T21:58:57.981785+0000","last_peered":"2024-04-23T21:58:57.981785+0000","last_clean":"2024-04-23T21:58:57.981785+0000","last_became_active":"2024-04-23T21:19:03.617420+0000","last_became_peered":"2024-04-23T21:19:03.617420+0000","last_unstale":"2024-04-23T21:58:57.981785+0000","last_undegraded":"2024-04-23T21:58:57.981785+0000","last_fullsized":"2024-04-23T21:58:57.981785+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:10.514458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.504986+0000","last_change":"2024-04-23T21:19:03.616216+0000","last_active":"2024-04-23T21:58:57.504986+0000","last_peered":"2024-04-23T21:58:57.504986+0000","last_clean":"2024-04-23T21:58:57.504986+0000","last_became_active":"2024-04-23T21:19:03.615832+0000","last_became_peered":"2024-04-23T21:19:03.615832+0000","last_unstale":"2024-04-23T21:58:57.504986+0000","last_undegraded":"2024-04-23T21:58:57.504986+0000","last_fullsized":"2024-04-23T21:58:57.504986+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:25:34.897612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":947,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.507083+0000","last_change":"2024-04-23T21:18:58.602249+0000","last_active":"2024-04-23T21:58:57.507083+0000","last_peered":"2024-04-23T21:58:57.507083+0000","last_clean":"2024-04-23T21:58:57.507083+0000","last_became_active":"2024-04-23T21:18:58.600198+0000","last_became_peered":"2024-04-23T21:18:58.600198+0000","last_unstale":"2024-04-23T21:58:57.507083+0000","last_undegraded":"2024-04-23T21:58:57.507083+0000","last_fullsized":"2024-04-23T21:58:57.507083+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:57.592214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:57.592214+0000","last_clean_scrub_stamp":"2024-04-23T21:18:57.592214+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:36:39.822147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":5808,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738855,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1232236,"kb_used_data":4768,"kb_used_omap":141,"kb_used_meta":1227314,"kb_avail":93139604,"statfs":{"total":96636764160,"available":95374954496,"internally_reserved":0,"allocated":4882432,"data_stored":2840926,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":145248,"internal_metadata":1256769696},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738856,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":894708,"kb_used_data":4712,"kb_used_omap":180,"kb_used_meta":889803,"kb_avail":93477132,"statfs":{"total":96636764160,"available":95720583168,"internally_reserved":0,"allocated":4825088,"data_stored":2250558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":184673,"internal_metadata":911158943},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738855,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":980908,"kb_used_data":4768,"kb_used_omap":244,"kb_used_meta":975883,"kb_avail":93390932,"statfs":{"total":96636764160,"available":95632314368,"internally_reserved":0,"allocated":4882432,"data_stored":2840926,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":249883,"internal_metadata":999305189},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T21:59:10.641 INFO:tasks.ceph.ceph_manager.cluster2:clean! 2024-04-23T21:59:10.641 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-04-23T21:59:10.920 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:59:10.920 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-23T21:59:10.933 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":1610,"stamp":"2024-04-23T21:59:09.102026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3107852,"kb_used_data":14248,"kb_used_omap":566,"kb_used_meta":3093001,"kb_avail":280007668,"statfs":{"total":289910292480,"available":286727852032,"internally_reserved":0,"allocated":14589952,"data_stored":7932410,"data_compressed":5808,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":579804,"internal_metadata":3167233828},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001652"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":812,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.508336+0000","last_change":"2024-04-23T21:19:03.617238+0000","last_active":"2024-04-23T21:58:57.508336+0000","last_peered":"2024-04-23T21:58:57.508336+0000","last_clean":"2024-04-23T21:58:57.508336+0000","last_became_active":"2024-04-23T21:19:03.616645+0000","last_became_peered":"2024-04-23T21:19:03.616645+0000","last_unstale":"2024-04-23T21:58:57.508336+0000","last_undegraded":"2024-04-23T21:58:57.508336+0000","last_fullsized":"2024-04-23T21:58:57.508336+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:25.085268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":812,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.507764+0000","last_change":"2024-04-23T21:19:03.617657+0000","last_active":"2024-04-23T21:58:57.507764+0000","last_peered":"2024-04-23T21:58:57.507764+0000","last_clean":"2024-04-23T21:58:57.507764+0000","last_became_active":"2024-04-23T21:19:03.616675+0000","last_became_peered":"2024-04-23T21:19:03.616675+0000","last_unstale":"2024-04-23T21:58:57.507764+0000","last_undegraded":"2024-04-23T21:58:57.507764+0000","last_fullsized":"2024-04-23T21:58:57.507764+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:11.044595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.508313+0000","last_change":"2024-04-23T21:19:03.617922+0000","last_active":"2024-04-23T21:58:57.508313+0000","last_peered":"2024-04-23T21:58:57.508313+0000","last_clean":"2024-04-23T21:58:57.508313+0000","last_became_active":"2024-04-23T21:19:03.616607+0000","last_became_peered":"2024-04-23T21:19:03.616607+0000","last_unstale":"2024-04-23T21:58:57.508313+0000","last_undegraded":"2024-04-23T21:58:57.508313+0000","last_fullsized":"2024-04-23T21:58:57.508313+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:36:08.192457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.504113+0000","last_change":"2024-04-23T21:19:03.616894+0000","last_active":"2024-04-23T21:58:57.504113+0000","last_peered":"2024-04-23T21:58:57.504113+0000","last_clean":"2024-04-23T21:58:57.504113+0000","last_became_active":"2024-04-23T21:19:03.616601+0000","last_became_peered":"2024-04-23T21:19:03.616601+0000","last_unstale":"2024-04-23T21:58:57.504113+0000","last_undegraded":"2024-04-23T21:58:57.504113+0000","last_fullsized":"2024-04-23T21:58:57.504113+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:46:19.804032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":812,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:56.506788+0000","last_change":"2024-04-23T21:19:03.615613+0000","last_active":"2024-04-23T21:58:56.506788+0000","last_peered":"2024-04-23T21:58:56.506788+0000","last_clean":"2024-04-23T21:58:56.506788+0000","last_became_active":"2024-04-23T21:19:03.615283+0000","last_became_peered":"2024-04-23T21:19:03.615283+0000","last_unstale":"2024-04-23T21:58:56.506788+0000","last_undegraded":"2024-04-23T21:58:56.506788+0000","last_fullsized":"2024-04-23T21:58:56.506788+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:14:30.533352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.981208+0000","last_change":"2024-04-23T21:19:03.617229+0000","last_active":"2024-04-23T21:58:57.981208+0000","last_peered":"2024-04-23T21:58:57.981208+0000","last_clean":"2024-04-23T21:58:57.981208+0000","last_became_active":"2024-04-23T21:19:03.616884+0000","last_became_peered":"2024-04-23T21:19:03.616884+0000","last_unstale":"2024-04-23T21:58:57.981208+0000","last_undegraded":"2024-04-23T21:58:57.981208+0000","last_fullsized":"2024-04-23T21:58:57.981208+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:11:38.295053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.981785+0000","last_change":"2024-04-23T21:19:03.617745+0000","last_active":"2024-04-23T21:58:57.981785+0000","last_peered":"2024-04-23T21:58:57.981785+0000","last_clean":"2024-04-23T21:58:57.981785+0000","last_became_active":"2024-04-23T21:19:03.617420+0000","last_became_peered":"2024-04-23T21:19:03.617420+0000","last_unstale":"2024-04-23T21:58:57.981785+0000","last_undegraded":"2024-04-23T21:58:57.981785+0000","last_fullsized":"2024-04-23T21:58:57.981785+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:10.514458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.504986+0000","last_change":"2024-04-23T21:19:03.616216+0000","last_active":"2024-04-23T21:58:57.504986+0000","last_peered":"2024-04-23T21:58:57.504986+0000","last_clean":"2024-04-23T21:58:57.504986+0000","last_became_active":"2024-04-23T21:19:03.615832+0000","last_became_peered":"2024-04-23T21:19:03.615832+0000","last_unstale":"2024-04-23T21:58:57.504986+0000","last_undegraded":"2024-04-23T21:58:57.504986+0000","last_fullsized":"2024-04-23T21:58:57.504986+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:25:34.897612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":947,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.507083+0000","last_change":"2024-04-23T21:18:58.602249+0000","last_active":"2024-04-23T21:58:57.507083+0000","last_peered":"2024-04-23T21:58:57.507083+0000","last_clean":"2024-04-23T21:58:57.507083+0000","last_became_active":"2024-04-23T21:18:58.600198+0000","last_became_peered":"2024-04-23T21:18:58.600198+0000","last_unstale":"2024-04-23T21:58:57.507083+0000","last_undegraded":"2024-04-23T21:58:57.507083+0000","last_fullsized":"2024-04-23T21:58:57.507083+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:57.592214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:57.592214+0000","last_clean_scrub_stamp":"2024-04-23T21:18:57.592214+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:36:39.822147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":5808,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738855,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1232236,"kb_used_data":4768,"kb_used_omap":141,"kb_used_meta":1227314,"kb_avail":93139604,"statfs":{"total":96636764160,"available":95374954496,"internally_reserved":0,"allocated":4882432,"data_stored":2840926,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":145248,"internal_metadata":1256769696},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738856,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":894708,"kb_used_data":4712,"kb_used_omap":180,"kb_used_meta":889803,"kb_avail":93477132,"statfs":{"total":96636764160,"available":95720583168,"internally_reserved":0,"allocated":4825088,"data_stored":2250558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":184673,"internal_metadata":911158943},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738855,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":980908,"kb_used_data":4768,"kb_used_omap":244,"kb_used_meta":975883,"kb_avail":93390932,"statfs":{"total":96636764160,"available":95632314368,"internally_reserved":0,"allocated":4882432,"data_stored":2840926,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":249883,"internal_metadata":999305189},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T21:59:10.934 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2024-04-23T21:59:11.218 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:59:11.218 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":406,"fsid":"04592e9d-e1b9-4919-8817-70bceab06a1f","created":"2024-04-23T21:18:47.128102+0000","modified":"2024-04-23T21:58:56.498811+0000","last_up_change":"2024-04-23T21:18:55.580476+0000","last_in_change":"2024-04-23T21:18:47.958645+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T21:18:56.733503+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T21:19:02.110649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2321b5c-7aac-4fe7-9ec6-30b635f38c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":4253773404},{"type":"v1","addr":"172.21.15.150:6801","nonce":4253773404}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6803","nonce":4253773404},{"type":"v1","addr":"172.21.15.150:6805","nonce":4253773404}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6811","nonce":4253773404},{"type":"v1","addr":"172.21.15.150:6813","nonce":4253773404}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6807","nonce":4253773404},{"type":"v1","addr":"172.21.15.150:6809","nonce":4253773404}]},"public_addr":"172.21.15.150:6801/4253773404","cluster_addr":"172.21.15.150:6805/4253773404","heartbeat_back_addr":"172.21.15.150:6813/4253773404","heartbeat_front_addr":"172.21.15.150:6809/4253773404","state":["exists","up"]},{"osd":1,"uuid":"fad0d87f-9151-4aec-859f-1d499f45e002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":1557391161},{"type":"v1","addr":"172.21.15.150:6804","nonce":1557391161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":1557391161},{"type":"v1","addr":"172.21.15.150:6808","nonce":1557391161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":1557391161},{"type":"v1","addr":"172.21.15.150:6815","nonce":1557391161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":1557391161},{"type":"v1","addr":"172.21.15.150:6812","nonce":1557391161}]},"public_addr":"172.21.15.150:6804/1557391161","cluster_addr":"172.21.15.150:6808/1557391161","heartbeat_back_addr":"172.21.15.150:6815/1557391161","heartbeat_front_addr":"172.21.15.150:6812/1557391161","state":["exists","up"]},{"osd":2,"uuid":"b14908e0-1927-45f6-868a-32938df0ef0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":2990791785},{"type":"v1","addr":"172.21.15.150:6817","nonce":2990791785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":2990791785},{"type":"v1","addr":"172.21.15.150:6819","nonce":2990791785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":2990791785},{"type":"v1","addr":"172.21.15.150:6823","nonce":2990791785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":2990791785},{"type":"v1","addr":"172.21.15.150:6821","nonce":2990791785}]},"public_addr":"172.21.15.150:6817/2990791785","cluster_addr":"172.21.15.150:6819/2990791785","heartbeat_back_addr":"172.21.15.150:6823/2990791785","heartbeat_front_addr":"172.21.15.150:6821/2990791785","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:52.931355+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:53.031762+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:52.998297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.150:0/1384796232":"2024-04-23T22:58:44.219771+0000","172.21.15.150:0/3172401768":"2024-04-23T22:58:06.876873+0000","172.21.15.150:0/903872264":"2024-04-23T22:56:39.809186+0000","172.21.15.150:0/1408988284":"2024-04-23T22:55:54.554730+0000","172.21.15.150:0/4019526498":"2024-04-23T22:55:13.113619+0000","172.21.15.150:0/4064193414":"2024-04-23T22:53:45.932268+0000","172.21.15.150:0/2649817105":"2024-04-23T22:53:05.312110+0000","172.21.15.150:0/3370483435":"2024-04-23T22:43:02.216627+0000","172.21.15.150:0/2369690060":"2024-04-23T22:35:36.450877+0000","172.21.15.150:0/99107369":"2024-04-23T22:57:21.910533+0000","172.21.15.150:0/3048372351":"2024-04-23T22:42:50.901348+0000","172.21.15.150:0/2307101005":"2024-04-23T22:42:42.853880+0000","172.21.15.150:0/1349155424":"2024-04-23T22:41:48.360619+0000","172.21.15.150:0/962578171":"2024-04-23T22:41:26.783023+0000","172.21.15.150:0/3166743372":"2024-04-23T22:52:25.052058+0000","172.21.15.150:0/844237658":"2024-04-23T22:45:50.603528+0000","172.21.15.150:0/4142824853":"2024-04-23T22:50:06.898308+0000","172.21.15.150:0/200702317":"2024-04-23T22:41:13.034677+0000","172.21.15.150:0/1135262800":"2024-04-23T22:48:48.652317+0000","172.21.15.150:0/3875146144":"2024-04-23T22:42:27.720628+0000","172.21.15.150:0/126729003":"2024-04-23T22:21:31.065909+0000","172.21.15.150:0/3361417344":"2024-04-23T22:43:28.710281+0000","172.21.15.150:0/1320704058":"2024-04-23T22:41:36.972561+0000","172.21.15.150:0/196589693":"2024-04-23T22:35:17.986713+0000","172.21.15.150:0/3348857562":"2024-04-23T22:50:27.043502+0000","172.21.15.150:0/2680081041":"2024-04-23T22:42:34.601166+0000","172.21.15.150:0/1434118524":"2024-04-23T22:45:26.248685+0000","172.21.15.150:0/3193926325":"2024-04-23T22:36:06.367409+0000","172.21.15.150:0/1633491878":"2024-04-23T22:24:10.618228+0000","172.21.15.150:0/870265356":"2024-04-23T22:49:25.588397+0000","172.21.15.150:0/2935522571":"2024-04-23T22:42:00.513941+0000","172.21.15.150:0/2918329563":"2024-04-23T22:27:53.202500+0000","172.21.15.150:0/2519774657":"2024-04-23T22:44:23.889074+0000","172.21.15.150:0/3415933789":"2024-04-23T22:22:29.784041+0000","172.21.15.150:0/2673042649":"2024-04-23T22:35:13.610899+0000","172.21.15.150:0/2274824350":"2024-04-23T22:54:28.413693+0000","172.21.15.150:0/1710438302":"2024-04-23T22:49:50.376114+0000","172.21.15.150:0/650630467":"2024-04-23T22:24:58.549835+0000","172.21.15.150:0/2578160225":"2024-04-23T22:40:41.171202+0000","172.21.15.150:0/3003100615":"2024-04-23T22:30:39.919398+0000","172.21.15.150:0/2126062357":"2024-04-23T22:41:20.295319+0000","172.21.15.150:0/3019266658":"2024-04-23T22:49:18.532694+0000","172.21.15.150:0/1070437488":"2024-04-23T22:50:17.350867+0000","172.21.15.150:0/1409098672":"2024-04-23T22:28:39.285199+0000","172.21.15.150:0/3211053429":"2024-04-23T22:43:58.688175+0000","172.21.15.150:0/974947764":"2024-04-23T22:42:06.669765+0000","172.21.15.150:0/2298939263":"2024-04-23T22:48:36.367656+0000","172.21.15.150:0/3481356753":"2024-04-23T22:29:30.801023+0000","172.21.15.150:0/3500175561":"2024-04-23T22:43:15.446379+0000","172.21.15.150:0/3540592208":"2024-04-23T22:44:31.207684+0000","172.21.15.150:0/2274390361":"2024-04-23T22:47:56.806645+0000","172.21.15.150:0/4050528897":"2024-04-23T22:45:35.527145+0000","172.21.15.150:0/3645923870":"2024-04-23T22:47:42.188465+0000","172.21.15.150:0/2659967871":"2024-04-23T22:27:02.681871+0000","172.21.15.150:0/1255496102":"2024-04-23T22:46:01.956313+0000","172.21.15.150:0/2318110257":"2024-04-23T22:25:54.517439+0000","172.21.15.150:0/4148749963":"2024-04-23T22:46:07.163569+0000","172.21.15.150:0/3593251239":"2024-04-23T22:44:41.321844+0000","172.21.15.150:0/4150172585":"2024-04-23T22:46:17.022699+0000","172.21.15.150:0/4194142691":"2024-04-23T22:46:32.260254+0000","172.21.15.150:0/1342265360":"2024-04-23T22:47:02.604630+0000","172.21.15.150:0/1197762026":"2024-04-23T22:47:16.685954+0000","172.21.15.150:0/2965696575":"2024-04-23T22:42:14.931397+0000","172.21.15.150:0/440966738":"2024-04-23T22:47:32.380434+0000","172.21.15.150:0/2631758931":"2024-04-23T22:39:31.087713+0000","172.21.15.150:0/2666377851":"2024-04-23T22:48:12.743561+0000","172.21.15.150:0/1617411706":"2024-04-23T22:48:19.571682+0000","172.21.15.150:0/292981776":"2024-04-23T22:48:31.604805+0000","172.21.15.150:0/3102201549":"2024-04-23T22:49:03.137519+0000","172.21.15.150:0/3625343926":"2024-04-23T22:49:58.158632+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T21:59:12.232 INFO:tasks.ceph:Scrubbing cluster2.osd.0 2024-04-23T21:59:12.232 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-23T21:59:12.356 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-23T21:59:12.356 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T21:59:12.356 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-23T21:59:12.367 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd deep-scrub 0 2024-04-23T21:59:12.653 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 0 to deep-scrub 2024-04-23T21:59:12.667 INFO:tasks.ceph:Scrubbing cluster2.osd.1 2024-04-23T21:59:12.667 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-23T21:59:12.788 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-23T21:59:12.788 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T21:59:12.788 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-23T21:59:12.799 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd deep-scrub 1 2024-04-23T21:59:13.097 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 1 to deep-scrub 2024-04-23T21:59:13.110 INFO:tasks.ceph:Scrubbing cluster2.osd.2 2024-04-23T21:59:13.110 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-23T21:59:13.231 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-23T21:59:13.231 INFO:teuthology.orchestra.run.smithi150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T21:59:13.231 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-23T21:59:13.242 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd deep-scrub 2 2024-04-23T21:59:13.527 INFO:teuthology.orchestra.run.smithi150.stderr:instructed osd(s) 2 to deep-scrub 2024-04-23T21:59:13.541 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-04-23T21:59:13.828 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:59:13.828 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-23T21:59:13.841 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":1612,"stamp":"2024-04-23T21:59:13.102731+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3107852,"kb_used_data":14248,"kb_used_omap":566,"kb_used_meta":3093001,"kb_avail":280007668,"statfs":{"total":289910292480,"available":286727852032,"internally_reserved":0,"allocated":14589952,"data_stored":7932410,"data_compressed":5808,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":579804,"internal_metadata":3167233828},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001722"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":812,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.508336+0000","last_change":"2024-04-23T21:19:03.617238+0000","last_active":"2024-04-23T21:58:57.508336+0000","last_peered":"2024-04-23T21:58:57.508336+0000","last_clean":"2024-04-23T21:58:57.508336+0000","last_became_active":"2024-04-23T21:19:03.616645+0000","last_became_peered":"2024-04-23T21:19:03.616645+0000","last_unstale":"2024-04-23T21:58:57.508336+0000","last_undegraded":"2024-04-23T21:58:57.508336+0000","last_fullsized":"2024-04-23T21:58:57.508336+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:25.085268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":812,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.507764+0000","last_change":"2024-04-23T21:19:03.617657+0000","last_active":"2024-04-23T21:58:57.507764+0000","last_peered":"2024-04-23T21:58:57.507764+0000","last_clean":"2024-04-23T21:58:57.507764+0000","last_became_active":"2024-04-23T21:19:03.616675+0000","last_became_peered":"2024-04-23T21:19:03.616675+0000","last_unstale":"2024-04-23T21:58:57.507764+0000","last_undegraded":"2024-04-23T21:58:57.507764+0000","last_fullsized":"2024-04-23T21:58:57.507764+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:49:11.044595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.508313+0000","last_change":"2024-04-23T21:19:03.617922+0000","last_active":"2024-04-23T21:58:57.508313+0000","last_peered":"2024-04-23T21:58:57.508313+0000","last_clean":"2024-04-23T21:58:57.508313+0000","last_became_active":"2024-04-23T21:19:03.616607+0000","last_became_peered":"2024-04-23T21:19:03.616607+0000","last_unstale":"2024-04-23T21:58:57.508313+0000","last_undegraded":"2024-04-23T21:58:57.508313+0000","last_fullsized":"2024-04-23T21:58:57.508313+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:36:08.192457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.504113+0000","last_change":"2024-04-23T21:19:03.616894+0000","last_active":"2024-04-23T21:58:57.504113+0000","last_peered":"2024-04-23T21:58:57.504113+0000","last_clean":"2024-04-23T21:58:57.504113+0000","last_became_active":"2024-04-23T21:19:03.616601+0000","last_became_peered":"2024-04-23T21:19:03.616601+0000","last_unstale":"2024-04-23T21:58:57.504113+0000","last_undegraded":"2024-04-23T21:58:57.504113+0000","last_fullsized":"2024-04-23T21:58:57.504113+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:46:19.804032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":812,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:56.506788+0000","last_change":"2024-04-23T21:19:03.615613+0000","last_active":"2024-04-23T21:58:56.506788+0000","last_peered":"2024-04-23T21:58:56.506788+0000","last_clean":"2024-04-23T21:58:56.506788+0000","last_became_active":"2024-04-23T21:19:03.615283+0000","last_became_peered":"2024-04-23T21:19:03.615283+0000","last_unstale":"2024-04-23T21:58:56.506788+0000","last_undegraded":"2024-04-23T21:58:56.506788+0000","last_fullsized":"2024-04-23T21:58:56.506788+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:14:30.533352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.981208+0000","last_change":"2024-04-23T21:19:03.617229+0000","last_active":"2024-04-23T21:58:57.981208+0000","last_peered":"2024-04-23T21:58:57.981208+0000","last_clean":"2024-04-23T21:58:57.981208+0000","last_became_active":"2024-04-23T21:19:03.616884+0000","last_became_peered":"2024-04-23T21:19:03.616884+0000","last_unstale":"2024-04-23T21:58:57.981208+0000","last_undegraded":"2024-04-23T21:58:57.981208+0000","last_fullsized":"2024-04-23T21:58:57.981208+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:11:38.295053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":814,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.981827+0000","last_change":"2024-04-23T21:19:03.617745+0000","last_active":"2024-04-23T21:58:57.981827+0000","last_peered":"2024-04-23T21:58:57.981827+0000","last_clean":"2024-04-23T21:58:57.981827+0000","last_became_active":"2024-04-23T21:19:03.617420+0000","last_became_peered":"2024-04-23T21:19:03.617420+0000","last_unstale":"2024-04-23T21:58:57.981827+0000","last_undegraded":"2024-04-23T21:58:57.981827+0000","last_fullsized":"2024-04-23T21:58:57.981827+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:51:10.514458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":813,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.504986+0000","last_change":"2024-04-23T21:19:03.616216+0000","last_active":"2024-04-23T21:58:57.504986+0000","last_peered":"2024-04-23T21:58:57.504986+0000","last_clean":"2024-04-23T21:58:57.504986+0000","last_became_active":"2024-04-23T21:19:03.615832+0000","last_became_peered":"2024-04-23T21:19:03.615832+0000","last_unstale":"2024-04-23T21:58:57.504986+0000","last_undegraded":"2024-04-23T21:58:57.504986+0000","last_fullsized":"2024-04-23T21:58:57.504986+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:19:02.601884+0000","last_clean_scrub_stamp":"2024-04-23T21:19:02.601884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:25:34.897612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":947,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:58:57.507083+0000","last_change":"2024-04-23T21:18:58.602249+0000","last_active":"2024-04-23T21:58:57.507083+0000","last_peered":"2024-04-23T21:58:57.507083+0000","last_clean":"2024-04-23T21:58:57.507083+0000","last_became_active":"2024-04-23T21:18:58.600198+0000","last_became_peered":"2024-04-23T21:18:58.600198+0000","last_unstale":"2024-04-23T21:58:57.507083+0000","last_undegraded":"2024-04-23T21:58:57.507083+0000","last_fullsized":"2024-04-23T21:58:57.507083+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:57.592214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:57.592214+0000","last_clean_scrub_stamp":"2024-04-23T21:18:57.592214+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:36:39.822147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":5808,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738856,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1232236,"kb_used_data":4768,"kb_used_omap":141,"kb_used_meta":1227314,"kb_avail":93139604,"statfs":{"total":96636764160,"available":95374954496,"internally_reserved":0,"allocated":4882432,"data_stored":2840926,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":145248,"internal_metadata":1256769696},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738857,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":894708,"kb_used_data":4712,"kb_used_omap":180,"kb_used_meta":889803,"kb_avail":93477132,"statfs":{"total":96636764160,"available":95720583168,"internally_reserved":0,"allocated":4825088,"data_stored":2250558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":184673,"internal_metadata":911158943},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738856,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":980908,"kb_used_data":4768,"kb_used_omap":244,"kb_used_meta":975883,"kb_avail":93390932,"statfs":{"total":96636764160,"available":95632314368,"internally_reserved":0,"allocated":4882432,"data_stored":2840926,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":249883,"internal_metadata":999305189},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T21:59:13.847 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-23T21:19:02.601884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=19, tm_sec=2, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=11, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:13.847 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-23T21:19:02.601884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=19, tm_sec=2, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=11, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:13.847 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-23T21:19:02.601884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=19, tm_sec=2, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=11, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:13.847 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-23T21:19:02.601884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=19, tm_sec=2, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=11, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:13.848 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-23T21:19:02.601884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=19, tm_sec=2, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=11, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:13.848 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-23T21:19:02.601884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=19, tm_sec=2, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=11, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:13.848 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-23T21:19:02.601884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=19, tm_sec=2, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=11, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:13.848 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-23T21:19:02.601884+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=19, tm_sec=2, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=11, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:13.848 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-23T21:18:57.592214+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=18, tm_sec=57, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=11, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:13.848 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-23T21:59:33.849 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2024-04-23T21:59:34.143 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-23T21:59:34.144 INFO:teuthology.orchestra.run.smithi150.stderr:dumped all 2024-04-23T21:59:34.156 INFO:teuthology.orchestra.run.smithi150.stdout:{"pg_ready":true,"pg_map":{"version":1622,"stamp":"2024-04-23T21:59:33.106794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3107840,"kb_used_data":14248,"kb_used_omap":566,"kb_used_meta":3093001,"kb_avail":280007680,"statfs":{"total":289910292480,"available":286727864320,"internally_reserved":0,"allocated":14589952,"data_stored":7932410,"data_compressed":5808,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":579804,"internal_metadata":3167233828},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002531"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":821,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:59:17.184691+0000","last_change":"2024-04-23T21:59:17.184712+0000","last_active":"2024-04-23T21:59:17.184691+0000","last_peered":"2024-04-23T21:59:17.184691+0000","last_clean":"2024-04-23T21:59:17.184691+0000","last_became_active":"2024-04-23T21:19:03.616645+0000","last_became_peered":"2024-04-23T21:19:03.616645+0000","last_unstale":"2024-04-23T21:59:17.184691+0000","last_undegraded":"2024-04-23T21:59:17.184691+0000","last_fullsized":"2024-04-23T21:59:17.184691+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:59:17.184599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:59:17.184599+0000","last_clean_scrub_stamp":"2024-04-23T21:59:17.184599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:55:16.627988+0000","scrub_duration":141,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":821,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:59:18.078629+0000","last_change":"2024-04-23T21:59:18.078653+0000","last_active":"2024-04-23T21:59:18.078629+0000","last_peered":"2024-04-23T21:59:18.078629+0000","last_clean":"2024-04-23T21:59:18.078629+0000","last_became_active":"2024-04-23T21:19:03.616675+0000","last_became_peered":"2024-04-23T21:19:03.616675+0000","last_unstale":"2024-04-23T21:59:18.078629+0000","last_undegraded":"2024-04-23T21:59:18.078629+0000","last_fullsized":"2024-04-23T21:59:18.078629+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:59:18.078506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:59:18.078506+0000","last_clean_scrub_stamp":"2024-04-23T21:59:18.078506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T09:36:28.031018+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":822,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:59:19.174753+0000","last_change":"2024-04-23T21:59:19.174777+0000","last_active":"2024-04-23T21:59:19.174753+0000","last_peered":"2024-04-23T21:59:19.174753+0000","last_clean":"2024-04-23T21:59:19.174753+0000","last_became_active":"2024-04-23T21:19:03.616607+0000","last_became_peered":"2024-04-23T21:19:03.616607+0000","last_unstale":"2024-04-23T21:59:19.174753+0000","last_undegraded":"2024-04-23T21:59:19.174753+0000","last_fullsized":"2024-04-23T21:59:19.174753+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:59:19.174654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:59:19.174654+0000","last_clean_scrub_stamp":"2024-04-23T21:59:19.174654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:38:12.827427+0000","scrub_duration":108,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":822,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:59:16.006912+0000","last_change":"2024-04-23T21:59:16.006933+0000","last_active":"2024-04-23T21:59:16.006912+0000","last_peered":"2024-04-23T21:59:16.006912+0000","last_clean":"2024-04-23T21:59:16.006912+0000","last_became_active":"2024-04-23T21:19:03.616601+0000","last_became_peered":"2024-04-23T21:19:03.616601+0000","last_unstale":"2024-04-23T21:59:16.006912+0000","last_undegraded":"2024-04-23T21:59:16.006912+0000","last_fullsized":"2024-04-23T21:59:16.006912+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:59:16.006822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:59:16.006822+0000","last_clean_scrub_stamp":"2024-04-23T21:59:16.006822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:23:27.078111+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":821,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:59:12.940510+0000","last_change":"2024-04-23T21:59:12.940536+0000","last_active":"2024-04-23T21:59:12.940510+0000","last_peered":"2024-04-23T21:59:12.940510+0000","last_clean":"2024-04-23T21:59:12.940510+0000","last_became_active":"2024-04-23T21:19:03.615283+0000","last_became_peered":"2024-04-23T21:19:03.615283+0000","last_unstale":"2024-04-23T21:59:12.940510+0000","last_undegraded":"2024-04-23T21:59:12.940510+0000","last_fullsized":"2024-04-23T21:59:12.940510+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:59:12.940414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:59:12.940414+0000","last_clean_scrub_stamp":"2024-04-23T21:59:12.940414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:54:04.536986+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":822,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:59:13.650351+0000","last_change":"2024-04-23T21:59:13.650372+0000","last_active":"2024-04-23T21:59:13.650351+0000","last_peered":"2024-04-23T21:59:13.650351+0000","last_clean":"2024-04-23T21:59:13.650351+0000","last_became_active":"2024-04-23T21:19:03.616884+0000","last_became_peered":"2024-04-23T21:19:03.616884+0000","last_unstale":"2024-04-23T21:59:13.650351+0000","last_undegraded":"2024-04-23T21:59:13.650351+0000","last_fullsized":"2024-04-23T21:59:13.650351+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:59:13.650251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:59:13.650251+0000","last_clean_scrub_stamp":"2024-04-23T21:59:13.650251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:54.836884+0000","scrub_duration":113,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":823,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:59:14.559361+0000","last_change":"2024-04-23T21:59:14.559383+0000","last_active":"2024-04-23T21:59:14.559361+0000","last_peered":"2024-04-23T21:59:14.559361+0000","last_clean":"2024-04-23T21:59:14.559361+0000","last_became_active":"2024-04-23T21:19:03.617420+0000","last_became_peered":"2024-04-23T21:19:03.617420+0000","last_unstale":"2024-04-23T21:59:14.559361+0000","last_undegraded":"2024-04-23T21:59:14.559361+0000","last_fullsized":"2024-04-23T21:59:14.559361+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:59:14.559267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:59:14.559267+0000","last_clean_scrub_stamp":"2024-04-23T21:59:14.559267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:50:20.381381+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":822,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:59:15.149193+0000","last_change":"2024-04-23T21:59:15.149214+0000","last_active":"2024-04-23T21:59:15.149193+0000","last_peered":"2024-04-23T21:59:15.149193+0000","last_clean":"2024-04-23T21:59:15.149193+0000","last_became_active":"2024-04-23T21:19:03.615832+0000","last_became_peered":"2024-04-23T21:19:03.615832+0000","last_unstale":"2024-04-23T21:59:15.149193+0000","last_undegraded":"2024-04-23T21:59:15.149193+0000","last_fullsized":"2024-04-23T21:59:15.149193+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:59:15.149098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:59:15.149098+0000","last_clean_scrub_stamp":"2024-04-23T21:59:15.149098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:04:25.222179+0000","scrub_duration":131,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"406'99","reported_seq":958,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-04-23T21:59:14.012427+0000","last_change":"2024-04-23T21:59:14.012476+0000","last_active":"2024-04-23T21:59:14.012427+0000","last_peered":"2024-04-23T21:59:14.012427+0000","last_clean":"2024-04-23T21:59:14.012427+0000","last_became_active":"2024-04-23T21:18:58.600198+0000","last_became_peered":"2024-04-23T21:18:58.600198+0000","last_unstale":"2024-04-23T21:59:14.012427+0000","last_undegraded":"2024-04-23T21:59:14.012427+0000","last_fullsized":"2024-04-23T21:59:14.012427+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'99","last_scrub_stamp":"2024-04-23T21:59:14.012323+0000","last_deep_scrub":"406'99","last_deep_scrub_stamp":"2024-04-23T21:59:14.012323+0000","last_clean_scrub_stamp":"2024-04-23T21:59:14.012323+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:10:43.983463+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":5808,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738860,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1232232,"kb_used_data":4768,"kb_used_omap":141,"kb_used_meta":1227314,"kb_avail":93139608,"statfs":{"total":96636764160,"available":95374958592,"internally_reserved":0,"allocated":4882432,"data_stored":2840926,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":145248,"internal_metadata":1256769696},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738861,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":894704,"kb_used_data":4712,"kb_used_omap":180,"kb_used_meta":889803,"kb_avail":93477136,"statfs":{"total":96636764160,"available":95720587264,"internally_reserved":0,"allocated":4825088,"data_stored":2250558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":184673,"internal_metadata":911158943},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738860,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":980904,"kb_used_data":4768,"kb_used_omap":244,"kb_used_meta":975883,"kb_avail":93390936,"statfs":{"total":96636764160,"available":95632318464,"internally_reserved":0,"allocated":4882432,"data_stored":2840926,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":249883,"internal_metadata":999305189},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2904,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T21:59:34.157 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster cluster2 config set global mon_health_to_clog false 2024-04-23T21:59:34.454 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-23T21:59:34.454 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-23T21:59:34.455 DEBUG:tasks.ceph.cluster2.osd.0:waiting for process to exit 2024-04-23T21:59:34.455 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T21:59:34.519 INFO:tasks.ceph.cluster2.osd.0:Stopped 2024-04-23T21:59:34.520 DEBUG:tasks.ceph.cluster2.osd.1:waiting for process to exit 2024-04-23T21:59:34.520 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T21:59:34.605 INFO:tasks.ceph.cluster2.osd.1:Stopped 2024-04-23T21:59:34.605 DEBUG:tasks.ceph.cluster2.osd.2:waiting for process to exit 2024-04-23T21:59:34.605 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T21:59:34.675 INFO:tasks.ceph.cluster2.osd.2:Stopped 2024-04-23T21:59:34.675 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-23T21:59:34.676 DEBUG:tasks.ceph.cluster2.mgr.x:waiting for process to exit 2024-04-23T21:59:34.676 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T21:59:37.031 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster2.mgr.x is failed for ~0s 2024-04-23T21:59:40.778 INFO:tasks.ceph.cluster2.mgr.x:Stopped 2024-04-23T21:59:40.778 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-23T21:59:40.778 DEBUG:tasks.ceph.cluster2.mon.a:waiting for process to exit 2024-04-23T21:59:40.778 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T21:59:40.795 INFO:tasks.ceph.cluster2.mon.a:Stopped 2024-04-23T21:59:40.795 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-23T21:59:40.795 DEBUG:teuthology.orchestra.run.smithi150:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/cluster2.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 2024-04-23T21:59:40.920 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster2-0 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-23T21:59:40.920 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/cluster2-0 2024-04-23T21:59:41.392 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster2-1 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-23T21:59:41.392 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/cluster2-1 2024-04-23T21:59:41.627 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster2-2 on ubuntu@smithi150.front.sepia.ceph.com 2024-04-23T21:59:41.628 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/cluster2-2 2024-04-23T21:59:41.854 INFO:tasks.ceph:Archiving mon data... 2024-04-23T21:59:41.856 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/mon/cluster2-a to /home/teuthworker/archive/teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi/7670210/data/cluster2.mon.a.tgz 2024-04-23T21:59:41.856 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-04-23T21:59:41.870 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.uIr5QpsmeP 2024-04-23T21:59:41.871 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar cz -f - -C /var/lib/ceph/mon/cluster2-a -- . > /tmp/tmp.uIr5QpsmeP 2024-04-23T21:59:42.032 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster2.mgr.x has been restored 2024-04-23T21:59:42.046 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.uIr5QpsmeP 2024-04-23T21:59:42.141 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.uIr5QpsmeP is 494KB 2024-04-23T21:59:42.210 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.uIr5QpsmeP 2024-04-23T21:59:42.225 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-23T21:59:42.225 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /etc/ceph/cluster2.conf /etc/ceph/cluster2.keyring /home/ubuntu/cephtest/cluster2.data /home/ubuntu/cephtest/cluster2.monmap /home/ubuntu/cephtest/../*.pid 2024-04-23T21:59:42.227 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /etc/ceph/cluster2.conf /etc/ceph/cluster2.keyring /home/ubuntu/cephtest/cluster2.data /home/ubuntu/cephtest/cluster2.monmap /home/ubuntu/cephtest/../*.pid 2024-04-23T21:59:42.426 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-23T21:59:42.437 INFO:tasks.ceph.ceph_manager.cluster1:waiting for clean 2024-04-23T21:59:42.437 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-04-23T21:59:42.992 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:59:42.992 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-23T21:59:43.005 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":1570,"stamp":"2024-04-23T21:59:42.222970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":4960212,"kb_used_data":8104,"kb_used_omap":700,"kb_used_meta":4951363,"kb_avail":278155308,"statfs":{"total":289910292480,"available":284831035392,"internally_reserved":0,"allocated":8298496,"data_stored":5509064,"data_compressed":5788,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":716936,"internal_metadata":5070196600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002503"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.612860+0000","last_change":"2024-04-23T21:18:27.927319+0000","last_active":"2024-04-23T21:58:55.612860+0000","last_peered":"2024-04-23T21:58:55.612860+0000","last_clean":"2024-04-23T21:58:55.612860+0000","last_became_active":"2024-04-23T21:18:27.926230+0000","last_became_peered":"2024-04-23T21:18:27.926230+0000","last_unstale":"2024-04-23T21:58:55.612860+0000","last_undegraded":"2024-04-23T21:58:55.612860+0000","last_fullsized":"2024-04-23T21:58:55.612860+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:16:09.753590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.610240+0000","last_change":"2024-04-23T21:18:27.929313+0000","last_active":"2024-04-23T21:58:55.610240+0000","last_peered":"2024-04-23T21:58:55.610240+0000","last_clean":"2024-04-23T21:58:55.610240+0000","last_became_active":"2024-04-23T21:18:27.928990+0000","last_became_peered":"2024-04-23T21:18:27.928990+0000","last_unstale":"2024-04-23T21:58:55.610240+0000","last_undegraded":"2024-04-23T21:58:55.610240+0000","last_fullsized":"2024-04-23T21:58:55.610240+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:13:49.198809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.610941+0000","last_change":"2024-04-23T21:18:27.926566+0000","last_active":"2024-04-23T21:58:55.610941+0000","last_peered":"2024-04-23T21:58:55.610941+0000","last_clean":"2024-04-23T21:58:55.610941+0000","last_became_active":"2024-04-23T21:18:27.926249+0000","last_became_peered":"2024-04-23T21:18:27.926249+0000","last_unstale":"2024-04-23T21:58:55.610941+0000","last_undegraded":"2024-04-23T21:58:55.610941+0000","last_fullsized":"2024-04-23T21:58:55.610941+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:11:22.863696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.613248+0000","last_change":"2024-04-23T21:18:27.926922+0000","last_active":"2024-04-23T21:58:55.613248+0000","last_peered":"2024-04-23T21:58:55.613248+0000","last_clean":"2024-04-23T21:58:55.613248+0000","last_became_active":"2024-04-23T21:18:27.926470+0000","last_became_peered":"2024-04-23T21:18:27.926470+0000","last_unstale":"2024-04-23T21:58:55.613248+0000","last_undegraded":"2024-04-23T21:58:55.613248+0000","last_fullsized":"2024-04-23T21:58:55.613248+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:18:14.714407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":663,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.285075+0000","last_change":"2024-04-23T21:18:27.929109+0000","last_active":"2024-04-23T21:58:55.285075+0000","last_peered":"2024-04-23T21:58:55.285075+0000","last_clean":"2024-04-23T21:58:55.285075+0000","last_became_active":"2024-04-23T21:18:27.928756+0000","last_became_peered":"2024-04-23T21:18:27.928756+0000","last_unstale":"2024-04-23T21:58:55.285075+0000","last_undegraded":"2024-04-23T21:58:55.285075+0000","last_fullsized":"2024-04-23T21:58:55.285075+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:26.553761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":663,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.613485+0000","last_change":"2024-04-23T21:18:27.929903+0000","last_active":"2024-04-23T21:58:55.613485+0000","last_peered":"2024-04-23T21:58:55.613485+0000","last_clean":"2024-04-23T21:58:55.613485+0000","last_became_active":"2024-04-23T21:18:27.929576+0000","last_became_peered":"2024-04-23T21:18:27.929576+0000","last_unstale":"2024-04-23T21:58:55.613485+0000","last_undegraded":"2024-04-23T21:58:55.613485+0000","last_fullsized":"2024-04-23T21:58:55.613485+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:18:11.837859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":664,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.606748+0000","last_change":"2024-04-23T21:18:27.929991+0000","last_active":"2024-04-23T21:58:55.606748+0000","last_peered":"2024-04-23T21:58:55.606748+0000","last_clean":"2024-04-23T21:58:55.606748+0000","last_became_active":"2024-04-23T21:18:27.929679+0000","last_became_peered":"2024-04-23T21:18:27.929679+0000","last_unstale":"2024-04-23T21:58:55.606748+0000","last_undegraded":"2024-04-23T21:58:55.606748+0000","last_fullsized":"2024-04-23T21:58:55.606748+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:34.885352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.614083+0000","last_change":"2024-04-23T21:18:27.926735+0000","last_active":"2024-04-23T21:58:55.614083+0000","last_peered":"2024-04-23T21:58:55.614083+0000","last_clean":"2024-04-23T21:58:55.614083+0000","last_became_active":"2024-04-23T21:18:27.926281+0000","last_became_peered":"2024-04-23T21:18:27.926281+0000","last_unstale":"2024-04-23T21:58:55.614083+0000","last_undegraded":"2024-04-23T21:58:55.614083+0000","last_fullsized":"2024-04-23T21:58:55.614083+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:17:26.651368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":796,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.609724+0000","last_change":"2024-04-23T21:18:22.904282+0000","last_active":"2024-04-23T21:58:55.609724+0000","last_peered":"2024-04-23T21:58:55.609724+0000","last_clean":"2024-04-23T21:58:55.609724+0000","last_became_active":"2024-04-23T21:18:22.903255+0000","last_became_peered":"2024-04-23T21:18:22.903255+0000","last_unstale":"2024-04-23T21:58:55.609724+0000","last_undegraded":"2024-04-23T21:58:55.609724+0000","last_fullsized":"2024-04-23T21:58:55.609724+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:21.893219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:21.893219+0000","last_clean_scrub_stamp":"2024-04-23T21:18:21.893219+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:17:55.065914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":5788,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738867,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1543224,"kb_used_data":2664,"kb_used_omap":183,"kb_used_meta":1540360,"kb_avail":92828616,"statfs":{"total":96636764160,"available":95056502784,"internally_reserved":0,"allocated":2727936,"data_stored":1442776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":187743,"internal_metadata":1577329313},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738867,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1811696,"kb_used_data":2720,"kb_used_omap":158,"kb_used_meta":1808801,"kb_avail":92560144,"statfs":{"total":96636764160,"available":94781587456,"internally_reserved":0,"allocated":2785280,"data_stored":2033144,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":162278,"internal_metadata":1852212762},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738870,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1605292,"kb_used_data":2720,"kb_used_omap":358,"kb_used_meta":1602201,"kb_avail":92766548,"statfs":{"total":96636764160,"available":94992945152,"internally_reserved":0,"allocated":2785280,"data_stored":2033144,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":366915,"internal_metadata":1640654525},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T21:59:43.005 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-04-23T21:59:43.285 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:59:43.285 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-23T21:59:43.297 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":1570,"stamp":"2024-04-23T21:59:42.222970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":4960212,"kb_used_data":8104,"kb_used_omap":700,"kb_used_meta":4951363,"kb_avail":278155308,"statfs":{"total":289910292480,"available":284831035392,"internally_reserved":0,"allocated":8298496,"data_stored":5509064,"data_compressed":5788,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":716936,"internal_metadata":5070196600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002503"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.612860+0000","last_change":"2024-04-23T21:18:27.927319+0000","last_active":"2024-04-23T21:58:55.612860+0000","last_peered":"2024-04-23T21:58:55.612860+0000","last_clean":"2024-04-23T21:58:55.612860+0000","last_became_active":"2024-04-23T21:18:27.926230+0000","last_became_peered":"2024-04-23T21:18:27.926230+0000","last_unstale":"2024-04-23T21:58:55.612860+0000","last_undegraded":"2024-04-23T21:58:55.612860+0000","last_fullsized":"2024-04-23T21:58:55.612860+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:16:09.753590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.610240+0000","last_change":"2024-04-23T21:18:27.929313+0000","last_active":"2024-04-23T21:58:55.610240+0000","last_peered":"2024-04-23T21:58:55.610240+0000","last_clean":"2024-04-23T21:58:55.610240+0000","last_became_active":"2024-04-23T21:18:27.928990+0000","last_became_peered":"2024-04-23T21:18:27.928990+0000","last_unstale":"2024-04-23T21:58:55.610240+0000","last_undegraded":"2024-04-23T21:58:55.610240+0000","last_fullsized":"2024-04-23T21:58:55.610240+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:13:49.198809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.610941+0000","last_change":"2024-04-23T21:18:27.926566+0000","last_active":"2024-04-23T21:58:55.610941+0000","last_peered":"2024-04-23T21:58:55.610941+0000","last_clean":"2024-04-23T21:58:55.610941+0000","last_became_active":"2024-04-23T21:18:27.926249+0000","last_became_peered":"2024-04-23T21:18:27.926249+0000","last_unstale":"2024-04-23T21:58:55.610941+0000","last_undegraded":"2024-04-23T21:58:55.610941+0000","last_fullsized":"2024-04-23T21:58:55.610941+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:11:22.863696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.613248+0000","last_change":"2024-04-23T21:18:27.926922+0000","last_active":"2024-04-23T21:58:55.613248+0000","last_peered":"2024-04-23T21:58:55.613248+0000","last_clean":"2024-04-23T21:58:55.613248+0000","last_became_active":"2024-04-23T21:18:27.926470+0000","last_became_peered":"2024-04-23T21:18:27.926470+0000","last_unstale":"2024-04-23T21:58:55.613248+0000","last_undegraded":"2024-04-23T21:58:55.613248+0000","last_fullsized":"2024-04-23T21:58:55.613248+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:18:14.714407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":663,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.285075+0000","last_change":"2024-04-23T21:18:27.929109+0000","last_active":"2024-04-23T21:58:55.285075+0000","last_peered":"2024-04-23T21:58:55.285075+0000","last_clean":"2024-04-23T21:58:55.285075+0000","last_became_active":"2024-04-23T21:18:27.928756+0000","last_became_peered":"2024-04-23T21:18:27.928756+0000","last_unstale":"2024-04-23T21:58:55.285075+0000","last_undegraded":"2024-04-23T21:58:55.285075+0000","last_fullsized":"2024-04-23T21:58:55.285075+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:26.553761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":663,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.613485+0000","last_change":"2024-04-23T21:18:27.929903+0000","last_active":"2024-04-23T21:58:55.613485+0000","last_peered":"2024-04-23T21:58:55.613485+0000","last_clean":"2024-04-23T21:58:55.613485+0000","last_became_active":"2024-04-23T21:18:27.929576+0000","last_became_peered":"2024-04-23T21:18:27.929576+0000","last_unstale":"2024-04-23T21:58:55.613485+0000","last_undegraded":"2024-04-23T21:58:55.613485+0000","last_fullsized":"2024-04-23T21:58:55.613485+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:18:11.837859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":664,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.606748+0000","last_change":"2024-04-23T21:18:27.929991+0000","last_active":"2024-04-23T21:58:55.606748+0000","last_peered":"2024-04-23T21:58:55.606748+0000","last_clean":"2024-04-23T21:58:55.606748+0000","last_became_active":"2024-04-23T21:18:27.929679+0000","last_became_peered":"2024-04-23T21:18:27.929679+0000","last_unstale":"2024-04-23T21:58:55.606748+0000","last_undegraded":"2024-04-23T21:58:55.606748+0000","last_fullsized":"2024-04-23T21:58:55.606748+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:34.885352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.614083+0000","last_change":"2024-04-23T21:18:27.926735+0000","last_active":"2024-04-23T21:58:55.614083+0000","last_peered":"2024-04-23T21:58:55.614083+0000","last_clean":"2024-04-23T21:58:55.614083+0000","last_became_active":"2024-04-23T21:18:27.926281+0000","last_became_peered":"2024-04-23T21:18:27.926281+0000","last_unstale":"2024-04-23T21:58:55.614083+0000","last_undegraded":"2024-04-23T21:58:55.614083+0000","last_fullsized":"2024-04-23T21:58:55.614083+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:17:26.651368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":796,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.609724+0000","last_change":"2024-04-23T21:18:22.904282+0000","last_active":"2024-04-23T21:58:55.609724+0000","last_peered":"2024-04-23T21:58:55.609724+0000","last_clean":"2024-04-23T21:58:55.609724+0000","last_became_active":"2024-04-23T21:18:22.903255+0000","last_became_peered":"2024-04-23T21:18:22.903255+0000","last_unstale":"2024-04-23T21:58:55.609724+0000","last_undegraded":"2024-04-23T21:58:55.609724+0000","last_fullsized":"2024-04-23T21:58:55.609724+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:21.893219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:21.893219+0000","last_clean_scrub_stamp":"2024-04-23T21:18:21.893219+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:17:55.065914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":5788,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738867,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1543224,"kb_used_data":2664,"kb_used_omap":183,"kb_used_meta":1540360,"kb_avail":92828616,"statfs":{"total":96636764160,"available":95056502784,"internally_reserved":0,"allocated":2727936,"data_stored":1442776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":187743,"internal_metadata":1577329313},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738867,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1811696,"kb_used_data":2720,"kb_used_omap":158,"kb_used_meta":1808801,"kb_avail":92560144,"statfs":{"total":96636764160,"available":94781587456,"internally_reserved":0,"allocated":2785280,"data_stored":2033144,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":162278,"internal_metadata":1852212762},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738870,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1605292,"kb_used_data":2720,"kb_used_omap":358,"kb_used_meta":1602201,"kb_avail":92766548,"statfs":{"total":96636764160,"available":94992945152,"internally_reserved":0,"allocated":2785280,"data_stored":2033144,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":366915,"internal_metadata":1640654525},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T21:59:43.298 INFO:tasks.ceph.ceph_manager.cluster1:clean! 2024-04-23T21:59:43.298 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-04-23T21:59:43.579 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:59:43.580 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-23T21:59:43.592 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":1570,"stamp":"2024-04-23T21:59:42.222970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":4960212,"kb_used_data":8104,"kb_used_omap":700,"kb_used_meta":4951363,"kb_avail":278155308,"statfs":{"total":289910292480,"available":284831035392,"internally_reserved":0,"allocated":8298496,"data_stored":5509064,"data_compressed":5788,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":716936,"internal_metadata":5070196600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002503"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.612860+0000","last_change":"2024-04-23T21:18:27.927319+0000","last_active":"2024-04-23T21:58:55.612860+0000","last_peered":"2024-04-23T21:58:55.612860+0000","last_clean":"2024-04-23T21:58:55.612860+0000","last_became_active":"2024-04-23T21:18:27.926230+0000","last_became_peered":"2024-04-23T21:18:27.926230+0000","last_unstale":"2024-04-23T21:58:55.612860+0000","last_undegraded":"2024-04-23T21:58:55.612860+0000","last_fullsized":"2024-04-23T21:58:55.612860+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:16:09.753590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.610240+0000","last_change":"2024-04-23T21:18:27.929313+0000","last_active":"2024-04-23T21:58:55.610240+0000","last_peered":"2024-04-23T21:58:55.610240+0000","last_clean":"2024-04-23T21:58:55.610240+0000","last_became_active":"2024-04-23T21:18:27.928990+0000","last_became_peered":"2024-04-23T21:18:27.928990+0000","last_unstale":"2024-04-23T21:58:55.610240+0000","last_undegraded":"2024-04-23T21:58:55.610240+0000","last_fullsized":"2024-04-23T21:58:55.610240+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:13:49.198809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.610941+0000","last_change":"2024-04-23T21:18:27.926566+0000","last_active":"2024-04-23T21:58:55.610941+0000","last_peered":"2024-04-23T21:58:55.610941+0000","last_clean":"2024-04-23T21:58:55.610941+0000","last_became_active":"2024-04-23T21:18:27.926249+0000","last_became_peered":"2024-04-23T21:18:27.926249+0000","last_unstale":"2024-04-23T21:58:55.610941+0000","last_undegraded":"2024-04-23T21:58:55.610941+0000","last_fullsized":"2024-04-23T21:58:55.610941+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:11:22.863696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.613248+0000","last_change":"2024-04-23T21:18:27.926922+0000","last_active":"2024-04-23T21:58:55.613248+0000","last_peered":"2024-04-23T21:58:55.613248+0000","last_clean":"2024-04-23T21:58:55.613248+0000","last_became_active":"2024-04-23T21:18:27.926470+0000","last_became_peered":"2024-04-23T21:18:27.926470+0000","last_unstale":"2024-04-23T21:58:55.613248+0000","last_undegraded":"2024-04-23T21:58:55.613248+0000","last_fullsized":"2024-04-23T21:58:55.613248+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:18:14.714407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":663,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.285075+0000","last_change":"2024-04-23T21:18:27.929109+0000","last_active":"2024-04-23T21:58:55.285075+0000","last_peered":"2024-04-23T21:58:55.285075+0000","last_clean":"2024-04-23T21:58:55.285075+0000","last_became_active":"2024-04-23T21:18:27.928756+0000","last_became_peered":"2024-04-23T21:18:27.928756+0000","last_unstale":"2024-04-23T21:58:55.285075+0000","last_undegraded":"2024-04-23T21:58:55.285075+0000","last_fullsized":"2024-04-23T21:58:55.285075+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:05:26.553761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":663,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.613485+0000","last_change":"2024-04-23T21:18:27.929903+0000","last_active":"2024-04-23T21:58:55.613485+0000","last_peered":"2024-04-23T21:58:55.613485+0000","last_clean":"2024-04-23T21:58:55.613485+0000","last_became_active":"2024-04-23T21:18:27.929576+0000","last_became_peered":"2024-04-23T21:18:27.929576+0000","last_unstale":"2024-04-23T21:58:55.613485+0000","last_undegraded":"2024-04-23T21:58:55.613485+0000","last_fullsized":"2024-04-23T21:58:55.613485+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:18:11.837859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":664,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.606748+0000","last_change":"2024-04-23T21:18:27.929991+0000","last_active":"2024-04-23T21:58:55.606748+0000","last_peered":"2024-04-23T21:58:55.606748+0000","last_clean":"2024-04-23T21:58:55.606748+0000","last_became_active":"2024-04-23T21:18:27.929679+0000","last_became_peered":"2024-04-23T21:18:27.929679+0000","last_unstale":"2024-04-23T21:58:55.606748+0000","last_undegraded":"2024-04-23T21:58:55.606748+0000","last_fullsized":"2024-04-23T21:58:55.606748+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:34.885352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.614083+0000","last_change":"2024-04-23T21:18:27.926735+0000","last_active":"2024-04-23T21:58:55.614083+0000","last_peered":"2024-04-23T21:58:55.614083+0000","last_clean":"2024-04-23T21:58:55.614083+0000","last_became_active":"2024-04-23T21:18:27.926281+0000","last_became_peered":"2024-04-23T21:18:27.926281+0000","last_unstale":"2024-04-23T21:58:55.614083+0000","last_undegraded":"2024-04-23T21:58:55.614083+0000","last_fullsized":"2024-04-23T21:58:55.614083+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:17:26.651368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":796,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.609724+0000","last_change":"2024-04-23T21:18:22.904282+0000","last_active":"2024-04-23T21:58:55.609724+0000","last_peered":"2024-04-23T21:58:55.609724+0000","last_clean":"2024-04-23T21:58:55.609724+0000","last_became_active":"2024-04-23T21:18:22.903255+0000","last_became_peered":"2024-04-23T21:18:22.903255+0000","last_unstale":"2024-04-23T21:58:55.609724+0000","last_undegraded":"2024-04-23T21:58:55.609724+0000","last_fullsized":"2024-04-23T21:58:55.609724+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:21.893219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:21.893219+0000","last_clean_scrub_stamp":"2024-04-23T21:18:21.893219+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:17:55.065914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":5788,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738867,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1543224,"kb_used_data":2664,"kb_used_omap":183,"kb_used_meta":1540360,"kb_avail":92828616,"statfs":{"total":96636764160,"available":95056502784,"internally_reserved":0,"allocated":2727936,"data_stored":1442776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":187743,"internal_metadata":1577329313},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738867,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1811696,"kb_used_data":2720,"kb_used_omap":158,"kb_used_meta":1808801,"kb_avail":92560144,"statfs":{"total":96636764160,"available":94781587456,"internally_reserved":0,"allocated":2785280,"data_stored":2033144,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":162278,"internal_metadata":1852212762},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738870,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1605292,"kb_used_data":2720,"kb_used_omap":358,"kb_used_meta":1602201,"kb_avail":92766548,"statfs":{"total":96636764160,"available":94992945152,"internally_reserved":0,"allocated":2785280,"data_stored":2033144,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":366915,"internal_metadata":1640654525},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T21:59:43.593 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2024-04-23T21:59:43.878 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:59:43.878 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":331,"fsid":"ed086417-ad7c-481e-9d20-547ac50d08e6","created":"2024-04-23T21:18:11.375874+0000","modified":"2024-04-23T21:58:55.279738+0000","last_up_change":"2024-04-23T21:18:19.688295+0000","last_in_change":"2024-04-23T21:18:12.261963+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T21:18:21.645859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-23T21:18:26.434320+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8341e94-a3c7-4414-938d-1eaac8b500b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":2042282222},{"type":"v1","addr":"172.21.15.148:6817","nonce":2042282222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":2042282222},{"type":"v1","addr":"172.21.15.148:6819","nonce":2042282222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":2042282222},{"type":"v1","addr":"172.21.15.148:6823","nonce":2042282222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":2042282222},{"type":"v1","addr":"172.21.15.148:6821","nonce":2042282222}]},"public_addr":"172.21.15.148:6817/2042282222","cluster_addr":"172.21.15.148:6819/2042282222","heartbeat_back_addr":"172.21.15.148:6823/2042282222","heartbeat_front_addr":"172.21.15.148:6821/2042282222","state":["exists","up"]},{"osd":1,"uuid":"b9bf3229-a5d5-4ad3-a48f-b5b73d1c32d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3232011875},{"type":"v1","addr":"172.21.15.148:6801","nonce":3232011875}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3232011875},{"type":"v1","addr":"172.21.15.148:6803","nonce":3232011875}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3232011875},{"type":"v1","addr":"172.21.15.148:6807","nonce":3232011875}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3232011875},{"type":"v1","addr":"172.21.15.148:6805","nonce":3232011875}]},"public_addr":"172.21.15.148:6801/3232011875","cluster_addr":"172.21.15.148:6803/3232011875","heartbeat_back_addr":"172.21.15.148:6807/3232011875","heartbeat_front_addr":"172.21.15.148:6805/3232011875","state":["exists","up"]},{"osd":2,"uuid":"1b13468d-227f-433d-88a3-4b93e4862340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":4099056617},{"type":"v1","addr":"172.21.15.148:6809","nonce":4099056617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":4099056617},{"type":"v1","addr":"172.21.15.148:6811","nonce":4099056617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":4099056617},{"type":"v1","addr":"172.21.15.148:6815","nonce":4099056617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":4099056617},{"type":"v1","addr":"172.21.15.148:6813","nonce":4099056617}]},"public_addr":"172.21.15.148:6809/4099056617","cluster_addr":"172.21.15.148:6811/4099056617","heartbeat_back_addr":"172.21.15.148:6815/4099056617","heartbeat_front_addr":"172.21.15.148:6813/4099056617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:17.448687+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:17.468223+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T21:18:17.467117+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":133,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T21:59:44.892 INFO:tasks.ceph:Scrubbing cluster1.osd.0 2024-04-23T21:59:44.892 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-23T21:59:45.025 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-23T21:59:45.025 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T21:59:45.025 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-23T21:59:45.036 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd deep-scrub 0 2024-04-23T21:59:45.320 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 0 to deep-scrub 2024-04-23T21:59:45.333 INFO:tasks.ceph:Scrubbing cluster1.osd.1 2024-04-23T21:59:45.333 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-23T21:59:45.472 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-23T21:59:45.472 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T21:59:45.472 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-23T21:59:45.482 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd deep-scrub 1 2024-04-23T21:59:45.772 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 1 to deep-scrub 2024-04-23T21:59:45.785 INFO:tasks.ceph:Scrubbing cluster1.osd.2 2024-04-23T21:59:45.786 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-23T21:59:45.907 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-04-23T21:59:45.907 INFO:teuthology.orchestra.run.smithi148.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-23T21:59:45.907 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-04-23T21:59:45.918 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd deep-scrub 2 2024-04-23T21:59:46.203 INFO:teuthology.orchestra.run.smithi148.stderr:instructed osd(s) 2 to deep-scrub 2024-04-23T21:59:46.216 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-04-23T21:59:46.500 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T21:59:46.501 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-23T21:59:46.513 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":1572,"stamp":"2024-04-23T21:59:46.223851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":4960212,"kb_used_data":8104,"kb_used_omap":700,"kb_used_meta":4951363,"kb_avail":278155308,"statfs":{"total":289910292480,"available":284831035392,"internally_reserved":0,"allocated":8298496,"data_stored":5509064,"data_compressed":5788,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":716936,"internal_metadata":5070196600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002536"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.612860+0000","last_change":"2024-04-23T21:18:27.927319+0000","last_active":"2024-04-23T21:58:55.612860+0000","last_peered":"2024-04-23T21:58:55.612860+0000","last_clean":"2024-04-23T21:58:55.612860+0000","last_became_active":"2024-04-23T21:18:27.926230+0000","last_became_peered":"2024-04-23T21:18:27.926230+0000","last_unstale":"2024-04-23T21:58:55.612860+0000","last_undegraded":"2024-04-23T21:58:55.612860+0000","last_fullsized":"2024-04-23T21:58:55.612860+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:16:09.753590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.610240+0000","last_change":"2024-04-23T21:18:27.929313+0000","last_active":"2024-04-23T21:58:55.610240+0000","last_peered":"2024-04-23T21:58:55.610240+0000","last_clean":"2024-04-23T21:58:55.610240+0000","last_became_active":"2024-04-23T21:18:27.928990+0000","last_became_peered":"2024-04-23T21:18:27.928990+0000","last_unstale":"2024-04-23T21:58:55.610240+0000","last_undegraded":"2024-04-23T21:58:55.610240+0000","last_fullsized":"2024-04-23T21:58:55.610240+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:13:49.198809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.610941+0000","last_change":"2024-04-23T21:18:27.926566+0000","last_active":"2024-04-23T21:58:55.610941+0000","last_peered":"2024-04-23T21:58:55.610941+0000","last_clean":"2024-04-23T21:58:55.610941+0000","last_became_active":"2024-04-23T21:18:27.926249+0000","last_became_peered":"2024-04-23T21:18:27.926249+0000","last_unstale":"2024-04-23T21:58:55.610941+0000","last_undegraded":"2024-04-23T21:58:55.610941+0000","last_fullsized":"2024-04-23T21:58:55.610941+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:11:22.863696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.613248+0000","last_change":"2024-04-23T21:18:27.926922+0000","last_active":"2024-04-23T21:58:55.613248+0000","last_peered":"2024-04-23T21:58:55.613248+0000","last_clean":"2024-04-23T21:58:55.613248+0000","last_became_active":"2024-04-23T21:18:27.926470+0000","last_became_peered":"2024-04-23T21:18:27.926470+0000","last_unstale":"2024-04-23T21:58:55.613248+0000","last_undegraded":"2024-04-23T21:58:55.613248+0000","last_fullsized":"2024-04-23T21:58:55.613248+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:18:14.714407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":668,"reported_epoch":331,"state":"active+clean+scrubbing+deep","last_fresh":"2024-04-23T21:59:45.343453+0000","last_change":"2024-04-23T21:59:45.343165+0000","last_active":"2024-04-23T21:59:45.343453+0000","last_peered":"2024-04-23T21:59:45.343453+0000","last_clean":"2024-04-23T21:59:45.343453+0000","last_became_active":"2024-04-23T21:18:27.928756+0000","last_became_peered":"2024-04-23T21:18:27.928756+0000","last_unstale":"2024-04-23T21:59:45.343453+0000","last_undegraded":"2024-04-23T21:59:45.343453+0000","last_fullsized":"2024-04-23T21:59:45.343453+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":663,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.613485+0000","last_change":"2024-04-23T21:18:27.929903+0000","last_active":"2024-04-23T21:58:55.613485+0000","last_peered":"2024-04-23T21:58:55.613485+0000","last_clean":"2024-04-23T21:58:55.613485+0000","last_became_active":"2024-04-23T21:18:27.929576+0000","last_became_peered":"2024-04-23T21:18:27.929576+0000","last_unstale":"2024-04-23T21:58:55.613485+0000","last_undegraded":"2024-04-23T21:58:55.613485+0000","last_fullsized":"2024-04-23T21:58:55.613485+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T08:18:11.837859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":664,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.606748+0000","last_change":"2024-04-23T21:18:27.929991+0000","last_active":"2024-04-23T21:58:55.606748+0000","last_peered":"2024-04-23T21:58:55.606748+0000","last_clean":"2024-04-23T21:58:55.606748+0000","last_became_active":"2024-04-23T21:18:27.929679+0000","last_became_peered":"2024-04-23T21:18:27.929679+0000","last_unstale":"2024-04-23T21:58:55.606748+0000","last_undegraded":"2024-04-23T21:58:55.606748+0000","last_fullsized":"2024-04-23T21:58:55.606748+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:52:34.885352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":662,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.614083+0000","last_change":"2024-04-23T21:18:27.926735+0000","last_active":"2024-04-23T21:58:55.614083+0000","last_peered":"2024-04-23T21:58:55.614083+0000","last_clean":"2024-04-23T21:58:55.614083+0000","last_became_active":"2024-04-23T21:18:27.926281+0000","last_became_peered":"2024-04-23T21:18:27.926281+0000","last_unstale":"2024-04-23T21:58:55.614083+0000","last_undegraded":"2024-04-23T21:58:55.614083+0000","last_fullsized":"2024-04-23T21:58:55.614083+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:26.903527+0000","last_clean_scrub_stamp":"2024-04-23T21:18:26.903527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:17:26.651368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":796,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:58:55.609724+0000","last_change":"2024-04-23T21:18:22.904282+0000","last_active":"2024-04-23T21:58:55.609724+0000","last_peered":"2024-04-23T21:58:55.609724+0000","last_clean":"2024-04-23T21:58:55.609724+0000","last_became_active":"2024-04-23T21:18:22.903255+0000","last_became_peered":"2024-04-23T21:18:22.903255+0000","last_unstale":"2024-04-23T21:58:55.609724+0000","last_undegraded":"2024-04-23T21:58:55.609724+0000","last_fullsized":"2024-04-23T21:58:55.609724+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:18:21.893219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:18:21.893219+0000","last_clean_scrub_stamp":"2024-04-23T21:18:21.893219+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:17:55.065914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":5788,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1543224,"kb_used_data":2664,"kb_used_omap":183,"kb_used_meta":1540360,"kb_avail":92828616,"statfs":{"total":96636764160,"available":95056502784,"internally_reserved":0,"allocated":2727936,"data_stored":1442776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":187743,"internal_metadata":1577329313},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738868,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1811696,"kb_used_data":2720,"kb_used_omap":158,"kb_used_meta":1808801,"kb_avail":92560144,"statfs":{"total":96636764160,"available":94781587456,"internally_reserved":0,"allocated":2785280,"data_stored":2033144,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":162278,"internal_metadata":1852212762},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738871,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1605292,"kb_used_data":2720,"kb_used_omap":358,"kb_used_meta":1602201,"kb_avail":92766548,"statfs":{"total":96636764160,"available":94992945152,"internally_reserved":0,"allocated":2785280,"data_stored":2033144,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":366915,"internal_metadata":1640654525},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T21:59:46.514 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-23T21:18:26.903527+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=18, tm_sec=26, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=43, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:46.514 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-23T21:18:26.903527+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=18, tm_sec=26, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=43, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:46.514 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-23T21:18:26.903527+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=18, tm_sec=26, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=43, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:46.514 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-23T21:18:26.903527+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=18, tm_sec=26, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=43, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:46.514 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-23T21:18:26.903527+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=18, tm_sec=26, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=43, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:46.515 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-23T21:18:26.903527+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=18, tm_sec=26, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=43, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:46.515 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-23T21:18:26.903527+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=18, tm_sec=26, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=43, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:46.515 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-23T21:18:26.903527+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=18, tm_sec=26, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=43, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:46.515 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-23T21:18:21.893219+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=18, tm_sec=21, tm_wday=1, tm_yday=114, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=23, tm_hour=21, tm_min=59, tm_sec=43, tm_wday=1, tm_yday=114, tm_isdst=0) 2024-04-23T21:59:46.515 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-23T22:00:06.516 DEBUG:teuthology.orchestra.run.smithi148:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2024-04-23T22:00:06.815 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-04-23T22:00:06.815 INFO:teuthology.orchestra.run.smithi148.stderr:dumped all 2024-04-23T22:00:06.828 INFO:teuthology.orchestra.run.smithi148.stdout:{"pg_ready":true,"pg_map":{"version":1582,"stamp":"2024-04-23T22:00:06.227383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":4960192,"kb_used_data":8104,"kb_used_omap":700,"kb_used_meta":4951363,"kb_avail":278155328,"statfs":{"total":289910292480,"available":284831055872,"internally_reserved":0,"allocated":8298496,"data_stored":5509064,"data_compressed":5788,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":716936,"internal_metadata":5070196600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002272"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":671,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:59:49.895924+0000","last_change":"2024-04-23T21:59:49.895976+0000","last_active":"2024-04-23T21:59:49.895924+0000","last_peered":"2024-04-23T21:59:49.895924+0000","last_clean":"2024-04-23T21:59:49.895924+0000","last_became_active":"2024-04-23T21:18:27.926230+0000","last_became_peered":"2024-04-23T21:18:27.926230+0000","last_unstale":"2024-04-23T21:59:49.895924+0000","last_undegraded":"2024-04-23T21:59:49.895924+0000","last_fullsized":"2024-04-23T21:59:49.895924+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:59:49.895769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:59:49.895769+0000","last_clean_scrub_stamp":"2024-04-23T21:59:49.895769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:11:10.304090+0000","scrub_duration":166,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":671,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:59:50.780019+0000","last_change":"2024-04-23T21:59:50.780059+0000","last_active":"2024-04-23T21:59:50.780019+0000","last_peered":"2024-04-23T21:59:50.780019+0000","last_clean":"2024-04-23T21:59:50.780019+0000","last_became_active":"2024-04-23T21:18:27.928990+0000","last_became_peered":"2024-04-23T21:18:27.928990+0000","last_unstale":"2024-04-23T21:59:50.780019+0000","last_undegraded":"2024-04-23T21:59:50.780019+0000","last_fullsized":"2024-04-23T21:59:50.780019+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:59:50.779893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:59:50.779893+0000","last_clean_scrub_stamp":"2024-04-23T21:59:50.779893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T07:15:21.005058+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":671,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:59:46.929124+0000","last_change":"2024-04-23T21:59:46.929163+0000","last_active":"2024-04-23T21:59:46.929124+0000","last_peered":"2024-04-23T21:59:46.929124+0000","last_clean":"2024-04-23T21:59:46.929124+0000","last_became_active":"2024-04-23T21:18:27.926249+0000","last_became_peered":"2024-04-23T21:18:27.926249+0000","last_unstale":"2024-04-23T21:59:46.929124+0000","last_undegraded":"2024-04-23T21:59:46.929124+0000","last_fullsized":"2024-04-23T21:59:46.929124+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:59:46.928973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:59:46.928973+0000","last_clean_scrub_stamp":"2024-04-23T21:59:46.928973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:42:02.104025+0000","scrub_duration":144,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":671,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:59:48.775256+0000","last_change":"2024-04-23T21:59:48.775278+0000","last_active":"2024-04-23T21:59:48.775256+0000","last_peered":"2024-04-23T21:59:48.775256+0000","last_clean":"2024-04-23T21:59:48.775256+0000","last_became_active":"2024-04-23T21:18:27.926470+0000","last_became_peered":"2024-04-23T21:18:27.926470+0000","last_unstale":"2024-04-23T21:59:48.775256+0000","last_undegraded":"2024-04-23T21:59:48.775256+0000","last_fullsized":"2024-04-23T21:59:48.775256+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:59:48.775173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:59:48.775173+0000","last_clean_scrub_stamp":"2024-04-23T21:59:48.775173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:38:10.588359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":673,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:59:45.419327+0000","last_change":"2024-04-23T21:59:45.419365+0000","last_active":"2024-04-23T21:59:45.419327+0000","last_peered":"2024-04-23T21:59:45.419327+0000","last_clean":"2024-04-23T21:59:45.419327+0000","last_became_active":"2024-04-23T21:18:27.928756+0000","last_became_peered":"2024-04-23T21:18:27.928756+0000","last_unstale":"2024-04-23T21:59:45.419327+0000","last_undegraded":"2024-04-23T21:59:45.419327+0000","last_fullsized":"2024-04-23T21:59:45.419327+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:59:45.419207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:59:45.419207+0000","last_clean_scrub_stamp":"2024-04-23T21:59:45.419207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:11:14.394696+0000","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":672,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:59:46.989662+0000","last_change":"2024-04-23T21:59:46.989684+0000","last_active":"2024-04-23T21:59:46.989662+0000","last_peered":"2024-04-23T21:59:46.989662+0000","last_clean":"2024-04-23T21:59:46.989662+0000","last_became_active":"2024-04-23T21:18:27.929576+0000","last_became_peered":"2024-04-23T21:18:27.929576+0000","last_unstale":"2024-04-23T21:59:46.989662+0000","last_undegraded":"2024-04-23T21:59:46.989662+0000","last_fullsized":"2024-04-23T21:59:46.989662+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:59:46.989566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:59:46.989566+0000","last_clean_scrub_stamp":"2024-04-23T21:59:46.989566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:21:15.969790+0000","scrub_duration":141,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":673,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:59:47.944495+0000","last_change":"2024-04-23T21:59:47.944518+0000","last_active":"2024-04-23T21:59:47.944495+0000","last_peered":"2024-04-23T21:59:47.944495+0000","last_clean":"2024-04-23T21:59:47.944495+0000","last_became_active":"2024-04-23T21:18:27.929679+0000","last_became_peered":"2024-04-23T21:18:27.929679+0000","last_unstale":"2024-04-23T21:59:47.944495+0000","last_undegraded":"2024-04-23T21:59:47.944495+0000","last_fullsized":"2024-04-23T21:59:47.944495+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:59:47.944401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:59:47.944401+0000","last_clean_scrub_stamp":"2024-04-23T21:59:47.944401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:35.003402+0000","scrub_duration":99,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":671,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:59:47.903087+0000","last_change":"2024-04-23T21:59:47.903108+0000","last_active":"2024-04-23T21:59:47.903087+0000","last_peered":"2024-04-23T21:59:47.903087+0000","last_clean":"2024-04-23T21:59:47.903087+0000","last_became_active":"2024-04-23T21:18:27.926281+0000","last_became_peered":"2024-04-23T21:18:27.926281+0000","last_unstale":"2024-04-23T21:59:47.903087+0000","last_undegraded":"2024-04-23T21:59:47.903087+0000","last_fullsized":"2024-04-23T21:59:47.903087+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T21:59:47.902976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T21:59:47.902976+0000","last_clean_scrub_stamp":"2024-04-23T21:59:47.902976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:46:43.085485+0000","scrub_duration":158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"331'99","reported_seq":807,"reported_epoch":331,"state":"active+clean","last_fresh":"2024-04-23T21:59:45.813828+0000","last_change":"2024-04-23T21:59:45.813878+0000","last_active":"2024-04-23T21:59:45.813828+0000","last_peered":"2024-04-23T21:59:45.813828+0000","last_clean":"2024-04-23T21:59:45.813828+0000","last_became_active":"2024-04-23T21:18:22.903255+0000","last_became_peered":"2024-04-23T21:18:22.903255+0000","last_unstale":"2024-04-23T21:59:45.813828+0000","last_undegraded":"2024-04-23T21:59:45.813828+0000","last_fullsized":"2024-04-23T21:59:45.813828+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"331'99","last_scrub_stamp":"2024-04-23T21:59:45.813726+0000","last_deep_scrub":"331'99","last_deep_scrub_stamp":"2024-04-23T21:59:45.813726+0000","last_clean_scrub_stamp":"2024-04-23T21:59:45.813726+0000","objects_scrubbed":2,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:47.403219+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":5788,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738872,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1543216,"kb_used_data":2664,"kb_used_omap":183,"kb_used_meta":1540360,"kb_avail":92828624,"statfs":{"total":96636764160,"available":95056510976,"internally_reserved":0,"allocated":2727936,"data_stored":1442776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":187743,"internal_metadata":1577329313},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738872,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1811688,"kb_used_data":2720,"kb_used_omap":158,"kb_used_meta":1808801,"kb_avail":92560152,"statfs":{"total":96636764160,"available":94781595648,"internally_reserved":0,"allocated":2785280,"data_stored":2033144,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":162278,"internal_metadata":1852212762},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738875,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1605288,"kb_used_data":2720,"kb_used_omap":358,"kb_used_meta":1602201,"kb_avail":92766552,"statfs":{"total":96636764160,"available":94992949248,"internally_reserved":0,"allocated":2785280,"data_stored":2033144,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":366915,"internal_metadata":1640654525},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":2894,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T22:00:06.829 DEBUG:teuthology.orchestra.run.smithi148:> sudo ceph --cluster cluster1 config set global mon_health_to_clog false 2024-04-23T22:00:07.129 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-23T22:00:07.129 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-23T22:00:07.130 DEBUG:tasks.ceph.cluster1.osd.0:waiting for process to exit 2024-04-23T22:00:07.130 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T22:00:07.191 INFO:tasks.ceph.cluster1.osd.0:Stopped 2024-04-23T22:00:07.191 DEBUG:tasks.ceph.cluster1.osd.1:waiting for process to exit 2024-04-23T22:00:07.191 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T22:00:07.271 INFO:tasks.ceph.cluster1.osd.1:Stopped 2024-04-23T22:00:07.271 DEBUG:tasks.ceph.cluster1.osd.2:waiting for process to exit 2024-04-23T22:00:07.271 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T22:00:07.351 INFO:tasks.ceph.cluster1.osd.2:Stopped 2024-04-23T22:00:07.352 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-23T22:00:07.352 DEBUG:tasks.ceph.cluster1.mgr.x:waiting for process to exit 2024-04-23T22:00:07.352 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T22:00:07.377 INFO:tasks.ceph.cluster1.mgr.x:Stopped 2024-04-23T22:00:07.378 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-23T22:00:07.378 DEBUG:tasks.ceph.cluster1.mon.a:waiting for process to exit 2024-04-23T22:00:07.378 INFO:teuthology.orchestra.run:waiting for 300 2024-04-23T22:00:07.430 INFO:tasks.ceph.cluster1.mon.a:Stopped 2024-04-23T22:00:07.430 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-23T22:00:07.430 DEBUG:teuthology.orchestra.run.smithi148:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/cluster1.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 2024-04-23T22:00:07.431 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster1.mon.a is failed for ~0s 2024-04-23T22:00:07.533 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster1-0 on ubuntu@smithi148.front.sepia.ceph.com 2024-04-23T22:00:07.533 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/cluster1-0 2024-04-23T22:00:07.939 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster1-1 on ubuntu@smithi148.front.sepia.ceph.com 2024-04-23T22:00:07.939 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/cluster1-1 2024-04-23T22:00:08.196 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster1-2 on ubuntu@smithi148.front.sepia.ceph.com 2024-04-23T22:00:08.197 DEBUG:teuthology.orchestra.run.smithi148:> sync && sudo umount -f /var/lib/ceph/osd/cluster1-2 2024-04-23T22:00:08.494 INFO:tasks.ceph:Archiving mon data... 2024-04-23T22:00:08.494 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/mon/cluster1-a to /home/teuthworker/archive/teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi/7670210/data/cluster1.mon.a.tgz 2024-04-23T22:00:08.495 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-04-23T22:00:08.510 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.M0HvhdeXLH 2024-04-23T22:00:08.510 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar cz -f - -C /var/lib/ceph/mon/cluster1-a -- . > /tmp/tmp.M0HvhdeXLH 2024-04-23T22:00:08.664 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.M0HvhdeXLH 2024-04-23T22:00:08.762 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.M0HvhdeXLH is 395KB 2024-04-23T22:00:08.858 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.M0HvhdeXLH 2024-04-23T22:00:08.873 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-23T22:00:08.873 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -rf -- /etc/ceph/cluster1.conf /etc/ceph/cluster1.keyring /home/ubuntu/cephtest/cluster1.data /home/ubuntu/cephtest/cluster1.monmap /home/ubuntu/cephtest/../*.pid 2024-04-23T22:00:08.914 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /etc/ceph/cluster1.conf /etc/ceph/cluster1.keyring /home/ubuntu/cephtest/cluster1.data /home/ubuntu/cephtest/cluster1.monmap /home/ubuntu/cephtest/../*.pid 2024-04-23T22:00:09.044 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-23T22:00:09.061 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-23T22:00:09.062 INFO:tasks.ceph:Archiving crash dumps... 2024-04-23T22:00:09.064 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi/7670210/remote/smithi148/crash 2024-04-23T22:00:09.065 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-23T22:00:09.107 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi/7670210/remote/smithi150/crash 2024-04-23T22:00:09.108 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-23T22:00:09.136 INFO:tasks.ceph:Compressing logs... 2024-04-23T22:00:09.136 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T22:00:09.153 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T22:00:09.202 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34254.log 2024-04-23T22:00:09.202 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34464.log 2024-04-23T22:00:09.202 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-mon.a.log 2024-04-23T22:00:09.203 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.34254.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34254.log.gz 2024-04-23T22:00:09.203 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.34464.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34464.log.gz 2024-04-23T22:00:09.203 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.30740.log 2024-04-23T22:00:09.203 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-mon.a.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35190.log 2024-04-23T22:00:09.203 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.30740.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.30813.log 2024-04-23T22:00:09.203 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.30740.log.gz 2024-04-23T22:00:09.203 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.35190.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35611.log 2024-04-23T22:00:09.203 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35190.log.gz 2024-04-23T22:00:09.203 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.30813.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35442.log 2024-04-23T22:00:09.203 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.30813.log.gz 2024-04-23T22:00:09.203 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.35611.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.25936.log 2024-04-23T22:00:09.203 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35611.log.gz 2024-04-23T22:00:09.203 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.35442.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34128.log 2024-04-23T22:00:09.203 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35442.log.gz 2024-04-23T22:00:09.204 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/ceph.tmp-client.admin.25936.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34041.log 2024-04-23T22:00:09.204 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.25936.log.gz 2024-04-23T22:00:09.204 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.34128.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35358.log 2024-04-23T22:00:09.204 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34128.log.gz 2024-04-23T22:00:09.204 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.34041.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34042.log 2024-04-23T22:00:09.204 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34041.log.gz 2024-04-23T22:00:09.204 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.35358.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34380.log 2024-04-23T22:00:09.204 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35358.log.gz 2024-04-23T22:00:09.204 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.34042.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.33792.log 2024-04-23T22:00:09.204 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34042.log.gz 2024-04-23T22:00:09.204 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.34380.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34293.log 2024-04-23T22:00:09.204 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34380.log.gz 2024-04-23T22:00:09.204 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.33792.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34040.log 2024-04-23T22:00:09.204 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.33792.log.gz 2024-04-23T22:00:09.204 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.34293.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34168.log 2024-04-23T22:00:09.204 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34293.log.gz 2024-04-23T22:00:09.205 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.34040.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.33960.log 2024-04-23T22:00:09.205 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34040.log.gz 2024-04-23T22:00:09.205 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.34168.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35232.log 2024-04-23T22:00:09.205 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34168.log.gz 2024-04-23T22:00:09.205 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.33960.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.30601.log 2024-04-23T22:00:09.205 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.33960.log.gz 2024-04-23T22:00:09.205 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.35232.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35148.log 2024-04-23T22:00:09.205 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35232.log.gz 2024-04-23T22:00:09.205 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.30601.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35274.log 2024-04-23T22:00:09.205 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.30601.log.gz 2024-04-23T22:00:09.205 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.35148.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35316.log 2024-04-23T22:00:09.205 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35148.log.gz 2024-04-23T22:00:09.205 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.35274.log: gzip -5 --verbose -- /var/log/ceph/cluster1.log 2024-04-23T22:00:09.205 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35274.log.gz 2024-04-23T22:00:09.205 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.35316.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35651.log 2024-04-23T22:00:09.206 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35316.log.gz 2024-04-23T22:00:09.206 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.33918.log 2024-04-23T22:00:09.206 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.35651.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34294.log 2024-04-23T22:00:09.206 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35651.log.gz 2024-04-23T22:00:09.206 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.33918.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.30403.log 2024-04-23T22:00:09.206 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.33918.log.gz 2024-04-23T22:00:09.206 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.34294.log: gzip -5 --verbose -- /var/log/ceph/cluster1-osd.0.log 2024-04-23T22:00:09.206 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34294.log.gz 2024-04-23T22:00:09.206 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.30403.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.30403.log.gz 2024-04-23T22:00:09.206 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34422.log 2024-04-23T22:00:09.206 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-osd.0.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.30641.log 2024-04-23T22:00:09.206 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.34422.log: gzip 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34422.log.gz -5 2024-04-23T22:00:09.206 INFO:teuthology.orchestra.run.smithi148.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.33652.log 2024-04-23T22:00:09.206 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.30641.log: gzip -5 --verbose -- /var/log/ceph/cluster1.audit.log 2024-04-23T22:00:09.206 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.30641.log.gz 2024-04-23T22:00:09.206 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.33652.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35568.log 2024-04-23T22:00:09.207 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.33652.log.gz 2024-04-23T22:00:09.207 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1.audit.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35400.log 2024-04-23T22:00:09.207 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.35568.log: gzip -5 --verbose -- /var/log/ceph/cluster2-mgr.x.log 2024-04-23T22:00:09.207 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35568.log.gz 2024-04-23T22:00:09.207 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.35400.log: gzip -5 --verbose -- /var/log/ceph/cluster1-osd.1.log 2024-04-23T22:00:09.207 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35400.log.gz 2024-04-23T22:00:09.207 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster2-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35526.log 2024-04-23T22:00:09.207 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-osd.1.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.30561.log 2024-04-23T22:00:09.207 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.35526.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.33736.log 2024-04-23T22:00:09.207 INFO:teuthology.orchestra.run.smithi148.stderr: 88.2% -- replaced with /var/log/ceph/cluster1.log.gz 2024-04-23T22:00:09.207 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35526.log.gz 2024-04-23T22:00:09.207 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.30561.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35484.log 2024-04-23T22:00:09.207 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.30561.log.gz 2024-04-23T22:00:09.207 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.33736.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.33876.log 2024-04-23T22:00:09.207 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.33736.log.gz 2024-04-23T22:00:09.208 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.35484.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.33834.log 2024-04-23T22:00:09.208 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35484.log.gz 2024-04-23T22:00:09.208 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.33876.log: gzip -5 --verbose -- /var/log/ceph/cluster1-mgr.x.log 2024-04-23T22:00:09.208 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.33876.log.gz 2024-04-23T22:00:09.208 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-client.admin.33834.log: gzip -5 --verbose -- /var/log/ceph/cluster1-osd.2.log 2024-04-23T22:00:09.208 INFO:teuthology.orchestra.run.smithi148.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.33834.log.gz 2024-04-23T22:00:09.208 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34167.log 2024-04-23T22:00:09.208 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cluster1-osd.2.log: /var/log/ceph/cluster1-client.admin.34167.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34167.log.gz 2024-04-23T22:00:09.231 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44414.log 2024-04-23T22:00:09.231 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102766.log 2024-04-23T22:00:09.232 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40523.log 2024-04-23T22:00:09.233 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44414.log: /var/log/ceph/cluster1-client.mirror.102766.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103607.log 2024-04-23T22:00:09.233 INFO:teuthology.orchestra.run.smithi150.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.102766.log.gz 2024-04-23T22:00:09.233 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.104707.log 2024-04-23T22:00:09.233 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40523.log: /var/log/ceph/cluster2-client.mirror.103607.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76850.log 2024-04-23T22:00:09.233 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.104707.log: 88.3%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.103607.log.gz -5 2024-04-23T22:00:09.233 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/cluster2-client.mirror.99593.log 2024-04-23T22:00:09.233 INFO:teuthology.orchestra.run.smithi150.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.104707.log.gz 2024-04-23T22:00:09.233 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49417.log 2024-04-23T22:00:09.238 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76850.log: /var/log/ceph/cluster2-client.mirror.99593.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34857.log 2024-04-23T22:00:09.239 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.76850.log.gz 2024-04-23T22:00:09.239 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.49417.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58072.log 2024-04-23T22:00:09.239 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34857.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93731.log 2024-04-23T22:00:09.239 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.99593.log.gz 2024-04-23T22:00:09.239 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34857.log.gz 2024-04-23T22:00:09.239 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58072.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58397.log 2024-04-23T22:00:09.239 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.93731.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59058.log 2024-04-23T22:00:09.239 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58397.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.52468.log 2024-04-23T22:00:09.239 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.93731.log.gz 2024-04-23T22:00:09.239 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.59058.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38898.log 2024-04-23T22:00:09.239 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.52468.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95766.log 2024-04-23T22:00:09.239 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.38898.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75505.log 2024-04-23T22:00:09.239 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.95766.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100847.log 2024-04-23T22:00:09.240 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75505.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48475.log 2024-04-23T22:00:09.240 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster1-client.mirror.95766.log.gz 2024-04-23T22:00:09.240 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100847.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59280.log 2024-04-23T22:00:09.240 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.75505.log.gz 2024-04-23T22:00:09.240 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48475.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.100847.log.gz 2024-04-23T22:00:09.240 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102293.log 2024-04-23T22:00:09.240 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.59280.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89967.log 2024-04-23T22:00:09.240 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102293.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.107404.log 2024-04-23T22:00:09.240 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102293.log.gz 2024-04-23T22:00:09.240 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.89967.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80207.log 2024-04-23T22:00:09.240 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.107404.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61391.log 2024-04-23T22:00:09.240 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.107404.log.gz 2024-04-23T22:00:09.240 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/cluster2-client.mirror.80207.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49650.log 2024-04-23T22:00:09.240 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.61391.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.80207.log.gz 2024-04-23T22:00:09.240 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56683.log 2024-04-23T22:00:09.241 INFO:teuthology.orchestra.run.smithi150.stderr: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.61391.log.gz 2024-04-23T22:00:09.241 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.49650.log: gzip 91.9% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.89967.log.gz --verbose 2024-04-23T22:00:09.241 INFO:teuthology.orchestra.run.smithi150.stderr: -- /var/log/ceph/cluster1-client.mirror.103126.log 2024-04-23T22:00:09.241 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56683.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50491.log 2024-04-23T22:00:09.348 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103126.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92078.log 2024-04-23T22:00:09.348 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.103126.log.gz 2024-04-23T22:00:09.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50491.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98291.log 2024-04-23T22:00:09.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.92078.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103667.log 2024-04-23T22:00:09.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98291.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.92078.log.gz 2024-04-23T22:00:09.349 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64037.log 2024-04-23T22:00:09.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.103667.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50828.log 2024-04-23T22:00:09.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.64037.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106441.log 2024-04-23T22:00:09.349 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.103667.log.gz 2024-04-23T22:00:09.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.50828.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73582.log 2024-04-23T22:00:09.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106441.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82170.log 2024-04-23T22:00:09.349 INFO:teuthology.orchestra.run.smithi150.stderr: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.98291.log.gz 2024-04-23T22:00:09.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73582.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.106441.log.gz 2024-04-23T22:00:09.349 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40426.log 2024-04-23T22:00:09.349 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.73582.log.gz 2024-04-23T22:00:09.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.82170.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94129.log 2024-04-23T22:00:09.350 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.82170.log.gz 2024-04-23T22:00:09.350 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40426.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74724.log 2024-04-23T22:00:09.350 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94129.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92880.log 2024-04-23T22:00:09.350 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74724.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.94129.log.gz 2024-04-23T22:00:09.350 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86787.log 2024-04-23T22:00:09.350 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.74724.log.gz 2024-04-23T22:00:09.350 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.92880.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45942.log 2024-04-23T22:00:09.350 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.92880.log.gz 2024-04-23T22:00:09.350 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86787.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102046.log 2024-04-23T22:00:09.350 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9%/var/log/ceph/cluster2-client.mirror.45942.log: -- replaced with /var/log/ceph/cluster2-client.mirror.86787.log.gz 2024-04-23T22:00:09.350 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43799.log 2024-04-23T22:00:09.350 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102046.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36070.log 2024-04-23T22:00:09.350 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102046.log.gz 2024-04-23T22:00:09.350 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose --/var/log/ceph/cluster2-client.mirror.43799.log: /var/log/ceph/cluster2-client.mirror.97890.log 2024-04-23T22:00:09.351 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.36070.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85769.log 2024-04-23T22:00:09.351 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97890.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95886.log 2024-04-23T22:00:09.351 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85769.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90525.log 2024-04-23T22:00:09.351 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.97890.log.gz 2024-04-23T22:00:09.351 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.85769.log.gz 2024-04-23T22:00:09.351 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95886.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38187.log 2024-04-23T22:00:09.351 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.95886.log.gz 2024-04-23T22:00:09.351 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90525.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99119.log 2024-04-23T22:00:09.351 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.90525.log.gz 2024-04-23T22:00:09.351 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.38187.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79740.log 2024-04-23T22:00:09.351 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99119.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72605.log 2024-04-23T22:00:09.351 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.79740.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101303.log 2024-04-23T22:00:09.351 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.72605.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.79740.log.gz 89.2% 2024-04-23T22:00:09.351 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.99119.log.gz 2024-04-23T22:00:09.351 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84394.log 2024-04-23T22:00:09.352 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.72605.log.gz 2024-04-23T22:00:09.352 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101303.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58246.log 2024-04-23T22:00:09.352 INFO:teuthology.orchestra.run.smithi150.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.101303.log.gz 2024-04-23T22:00:09.352 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.84394.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80161.log 2024-04-23T22:00:09.352 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58246.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.84394.log.gz 2024-04-23T22:00:09.352 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.107698.log 2024-04-23T22:00:09.352 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.80161.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94889.log 2024-04-23T22:00:09.352 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.80161.log.gz 2024-04-23T22:00:09.352 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.107698.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79212.log 2024-04-23T22:00:09.352 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.107698.log.gz 2024-04-23T22:00:09.352 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94889.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76403.log 2024-04-23T22:00:09.352 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.79212.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.94889.log.gz 2024-04-23T22:00:09.352 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104253.log 2024-04-23T22:00:09.352 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.79212.log.gz 2024-04-23T22:00:09.352 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76403.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77912.log 2024-04-23T22:00:09.353 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.76403.log.gz 2024-04-23T22:00:09.353 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104253.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.35193.log 2024-04-23T22:00:09.353 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.77912.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71636.log 2024-04-23T22:00:09.353 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.35193.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.35193.log.gz 2024-04-23T22:00:09.353 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105425.log 2024-04-23T22:00:09.353 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.77912.log.gz 2024-04-23T22:00:09.353 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71636.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102878.log 2024-04-23T22:00:09.353 INFO:teuthology.orchestra.run.smithi150.stderr: 91.3% -- replaced with /var/log/ceph/cluster2-client.mirror.104253.log.gz 2024-04-23T22:00:09.353 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105425.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66702.log 2024-04-23T22:00:09.353 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.105425.log.gz 2024-04-23T22:00:09.353 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102878.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74185.log 2024-04-23T22:00:09.353 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102878.log.gz 2024-04-23T22:00:09.353 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62975.log 2024-04-23T22:00:09.353 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66702.log: /var/log/ceph/cluster1-client.mirror.74185.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.107530.log 2024-04-23T22:00:09.353 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.66702.log.gz 2024-04-23T22:00:09.353 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.62975.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91498.log 2024-04-23T22:00:09.354 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.107530.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.107530.log.gz 2024-04-23T22:00:09.354 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.37813.log 2024-04-23T22:00:09.354 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.91498.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66100.log 2024-04-23T22:00:09.354 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.91498.log.gz 2024-04-23T22:00:09.354 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.37813.log: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.58246.log.gz 2024-04-23T22:00:09.354 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82275.log 2024-04-23T22:00:09.354 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57149.log 2024-04-23T22:00:09.354 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66100.log: /var/log/ceph/cluster2-client.mirror.82275.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87682.log 2024-04-23T22:00:09.354 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.66100.log.gz 2024-04-23T22:00:09.354 INFO:teuthology.orchestra.run.smithi150.stderr: 94.6%gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88901.log 2024-04-23T22:00:09.354 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.74185.log.gz/var/log/ceph/cluster2-client.mirror.57149.log: 2024-04-23T22:00:09.354 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87682.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61784.log 2024-04-23T22:00:09.354 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.87682.log.gz 2024-04-23T22:00:09.355 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.88901.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73085.log 2024-04-23T22:00:09.355 INFO:teuthology.orchestra.run.smithi150.stderr: 92.2%/var/log/ceph/cluster2-client.mirror.61784.log: -- replaced with /var/log/ceph/cluster2-client.mirror.82275.log.gz 2024-04-23T22:00:09.355 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.88901.log.gz 2024-04-23T22:00:09.355 INFO:teuthology.orchestra.run.smithi150.stderr: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.61784.log.gz 2024-04-23T22:00:09.355 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43896.log 2024-04-23T22:00:09.355 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73085.log: 88.0%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91942.log 2024-04-23T22:00:09.355 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.44414.log.gz 2024-04-23T22:00:09.355 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.43896.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85734.log 2024-04-23T22:00:09.355 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/cluster2-client.mirror.91942.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59587.log 2024-04-23T22:00:09.355 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85734.log: gzip 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.85734.log.gz 2024-04-23T22:00:09.355 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose 92.2% -- replaced with /var/log/ceph/cluster2-client.mirror.73085.log.gz 2024-04-23T22:00:09.355 INFO:teuthology.orchestra.run.smithi150.stderr: -- /var/log/ceph/cluster2-client.mirror.39747.log 2024-04-23T22:00:09.355 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.59587.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.45942.log.gz 2024-04-23T22:00:09.355 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.39747.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.38898.log.gz 2024-04-23T22:00:09.356 INFO:teuthology.orchestra.run.smithi150.stderr:gzip 56.7% -- replaced with /var/log/ceph/cluster1-client.mirror.36070.log.gz 2024-04-23T22:00:09.356 INFO:teuthology.orchestra.run.smithi150.stderr: -5 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.59280.log.gz 2024-04-23T22:00:09.356 INFO:teuthology.orchestra.run.smithi150.stderr: 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.58397.log.gz 2024-04-23T22:00:09.356 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.107090.log 2024-04-23T22:00:09.356 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60671.log 2024-04-23T22:00:09.356 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.107090.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.107090.log.gz 2024-04-23T22:00:09.356 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63839.log 2024-04-23T22:00:09.356 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99141.log 2024-04-23T22:00:09.356 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60671.log: /var/log/ceph/cluster2-client.mirror.63839.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56510.log 2024-04-23T22:00:09.356 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99141.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51143.log 2024-04-23T22:00:09.356 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56510.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.99141.log.gz 2024-04-23T22:00:09.356 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58418.log 2024-04-23T22:00:09.356 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.51143.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93020.log 2024-04-23T22:00:09.356 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58418.log: /var/log/ceph/cluster2-client.mirror.93020.log: 87.9% 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.58418.log.gz 2024-04-23T22:00:09.356 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.51143.log.gz 2024-04-23T22:00:09.357 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.93020.log.gz 2024-04-23T22:00:09.357 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87874.log 2024-04-23T22:00:09.357 INFO:teuthology.orchestra.run.smithi150.stderr: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.91942.log.gz 2024-04-23T22:00:09.357 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55638.log 2024-04-23T22:00:09.357 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58870.log 2024-04-23T22:00:09.357 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87874.log: /var/log/ceph/cluster2-client.mirror.55638.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100612.log 2024-04-23T22:00:09.357 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose --/var/log/ceph/cluster2-client.mirror.58870.log: /var/log/ceph/cluster2-client.mirror.75547.log 2024-04-23T22:00:09.357 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/cluster1-client.admin.52468.log.gz 2024-04-23T22:00:09.357 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100612.log: 89.0% 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56683.log.gz 2024-04-23T22:00:09.357 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.100612.log.gz 2024-04-23T22:00:09.357 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97774.log 2024-04-23T22:00:09.357 INFO:teuthology.orchestra.run.smithi150.stderr: 92.5% -- replaced with /var/log/ceph/cluster2-client.mirror.87874.log.gz 2024-04-23T22:00:09.357 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75547.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93901.log 2024-04-23T22:00:09.357 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97774.log: gzip 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.75547.log.gz 2024-04-23T22:00:09.357 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73744.log 2024-04-23T22:00:09.358 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.93901.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.44569.log 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93901.log.gz 2024-04-23T22:00:09.358 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.358 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73744.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69562.log 2024-04-23T22:00:09.358 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.73744.log.gz 2024-04-23T22:00:09.358 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.44569.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62251.log 2024-04-23T22:00:09.358 INFO:teuthology.orchestra.run.smithi150.stderr: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.97774.log.gz 2024-04-23T22:00:09.358 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.69562.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34269.log 2024-04-23T22:00:09.358 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.62251.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49188.log 2024-04-23T22:00:09.358 INFO:teuthology.orchestra.run.smithi150.stderr: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.62251.log.gz 2024-04-23T22:00:09.358 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.91882.log 2024-04-23T22:00:09.358 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34269.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34269.log.gz 2024-04-23T22:00:09.358 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.49188.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.43799.log.gz 2024-04-23T22:00:09.358 INFO:teuthology.orchestra.run.smithi150.stderr: 91.4% -- replaced with /var/log/ceph/cluster1-client.mirror.69562.log.gz 2024-04-23T22:00:09.359 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.43896.log.gz 2024-04-23T22:00:09.359 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54033.log 2024-04-23T22:00:09.359 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.91882.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98847.log 2024-04-23T22:00:09.359 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster2-client.mirror.54033.log: -- /var/log/ceph/cluster2-client.mirror.59241.log 2024-04-23T22:00:09.359 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98847.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66967.log 2024-04-23T22:00:09.359 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.59241.log: gzip -5 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.98847.log.gz 2024-04-23T22:00:09.359 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/cluster2-client.mirror.38015.log 2024-04-23T22:00:09.359 INFO:teuthology.orchestra.run.smithi150.stderr: 91.7% -- replaced with /var/log/ceph/cluster1-client.mirror.91882.log.gz 2024-04-23T22:00:09.359 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66967.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56468.log 2024-04-23T22:00:09.359 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.66967.log.gz/var/log/ceph/cluster2-client.mirror.38015.log: 2024-04-23T22:00:09.359 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41396.log 2024-04-23T22:00:09.359 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56468.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.3.45689.log 2024-04-23T22:00:09.359 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41396.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87108.log 2024-04-23T22:00:09.359 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.3.45689.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74545.log 2024-04-23T22:00:09.359 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87108.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76438.log 2024-04-23T22:00:09.360 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.87108.log.gz 2024-04-23T22:00:09.360 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74545.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51463.log 2024-04-23T22:00:09.360 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.74545.log.gz 2024-04-23T22:00:09.360 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76438.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82706.log 2024-04-23T22:00:09.360 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1%/var/log/ceph/cluster1-client.mirror.51463.log: -- replaced with /var/log/ceph/cluster2-client.mirror.76438.log.gz 2024-04-23T22:00:09.360 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76931.log 2024-04-23T22:00:09.360 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.82706.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.1.60061.log 2024-04-23T22:00:09.360 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.82706.log.gz 2024-04-23T22:00:09.360 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76931.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96617.log 2024-04-23T22:00:09.360 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.1.60061.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98032.log 2024-04-23T22:00:09.360 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.76931.log.gz 2024-04-23T22:00:09.360 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster2-client.mirror.96617.log: -- /var/log/ceph/cluster2-client.mirror.60535.log 2024-04-23T22:00:09.360 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98032.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.96617.log.gz 2024-04-23T22:00:09.360 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88266.log 2024-04-23T22:00:09.361 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.98032.log.gz 2024-04-23T22:00:09.361 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60535.log: /var/log/ceph/cluster1-client.mirror.88266.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53227.log 2024-04-23T22:00:09.361 INFO:teuthology.orchestra.run.smithi150.stderr:gzip 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.88266.log.gz 2024-04-23T22:00:09.361 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45988.log 2024-04-23T22:00:09.361 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.53227.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41361.log 95.9% -- replaced with /var/log/ceph/cluster2-client.mirror.71636.log.gz 2024-04-23T22:00:09.361 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.361 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.45988.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80718.log 2024-04-23T22:00:09.361 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41361.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73628.log 2024-04-23T22:00:09.361 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.80718.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72210.log 2024-04-23T22:00:09.361 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.80718.log.gz -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105847.log 2024-04-23T22:00:09.361 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.361 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73628.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65575.log 2024-04-23T22:00:09.361 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.73628.log.gz 2024-04-23T22:00:09.361 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.72210.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100512.log/var/log/ceph/cluster2-client.mirror.105847.log: 2024-04-23T22:00:09.361 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.72210.log.gz 2024-04-23T22:00:09.362 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.65575.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58029.log 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.65575.log.gz 2024-04-23T22:00:09.362 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.362 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100512.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89135.log 2024-04-23T22:00:09.362 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58029.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91197.log 2024-04-23T22:00:09.362 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.89135.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46888.log 2024-04-23T22:00:09.362 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster1-client.mirror.89135.log.gz/var/log/ceph/cluster2-client.mirror.91197.log: 2024-04-23T22:00:09.362 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100279.log 2024-04-23T22:00:09.362 INFO:teuthology.orchestra.run.smithi150.stderr: 90.5% 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.91197.log.gz 2024-04-23T22:00:09.362 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.100512.log.gzgzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102113.log 2024-04-23T22:00:09.362 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.362 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46888.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.54033.log.gz 2024-04-23T22:00:09.362 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41105.log 2024-04-23T22:00:09.362 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100279.log: /var/log/ceph/cluster1-client.mirror.102113.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76988.log 2024-04-23T22:00:09.362 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% 86.8% -- replaced with /var/log/ceph/cluster1-client.mirror.102113.log.gz 2024-04-23T22:00:09.363 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.100279.log.gz 2024-04-23T22:00:09.363 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41105.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58334.log 2024-04-23T22:00:09.363 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76988.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69542.log 2024-04-23T22:00:09.363 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58334.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.37914.log 2024-04-23T22:00:09.363 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69542.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76326.log 2024-04-23T22:00:09.363 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.37914.log: 91.7% -- replaced with /var/log/ceph/cluster2-client.mirror.76988.log.gz 2024-04-23T22:00:09.363 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46150.log 2024-04-23T22:00:09.363 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62008.log 2024-04-23T22:00:09.363 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76326.log: /var/log/ceph/cluster2-client.mirror.46150.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90405.log 2024-04-23T22:00:09.363 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.76326.log.gz 2024-04-23T22:00:09.363 INFO:teuthology.orchestra.run.smithi150.stderr: 91.5% -- replaced with /var/log/ceph/cluster2-client.mirror.69542.log.gz/var/log/ceph/cluster1-client.mirror.62008.log: 2024-04-23T22:00:09.363 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54908.log 2024-04-23T22:00:09.363 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90405.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65440.log 2024-04-23T22:00:09.363 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.54908.log: 87.9% 93.0% -- replaced with /var/log/ceph/cluster1-client.mirror.62008.log.gz 2024-04-23T22:00:09.363 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.90405.log.gz 2024-04-23T22:00:09.363 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57685.log 2024-04-23T22:00:09.364 INFO:teuthology.orchestra.run.smithi150.stderr: 84.5% -- replaced with /var/log/ceph/cluster2-client.mirror.59587.log.gz 2024-04-23T22:00:09.364 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48440.log 2024-04-23T22:00:09.364 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.65440.log: /var/log/ceph/cluster2-client.mirror.57685.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84761.log 2024-04-23T22:00:09.364 INFO:teuthology.orchestra.run.smithi150.stderr: 90.4% 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.65440.log.gz 2024-04-23T22:00:09.364 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.59058.log.gz 2024-04-23T22:00:09.364 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48440.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55083.log 2024-04-23T22:00:09.364 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.84761.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.45988.log.gz 2024-04-23T22:00:09.364 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.84761.log.gz 2024-04-23T22:00:09.364 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55083.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90355.log 2024-04-23T22:00:09.364 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46535.log 2024-04-23T22:00:09.364 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90355.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74510.log 2024-04-23T22:00:09.364 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.90355.log.gz 2024-04-23T22:00:09.364 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46535.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46217.log 2024-04-23T22:00:09.364 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74510.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67927.log 2024-04-23T22:00:09.364 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.74510.log.gz 2024-04-23T22:00:09.365 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.46217.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80473.log 2024-04-23T22:00:09.365 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5/var/log/ceph/cluster2-client.mirror.67927.log: --verbose -- /var/log/ceph/cluster2-client.mirror.47020.log 2024-04-23T22:00:09.365 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.80473.log: gzip -5 --verbose 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.80473.log.gz 2024-04-23T22:00:09.365 INFO:teuthology.orchestra.run.smithi150.stderr: -- /var/log/ceph/cluster2-client.mirror.94307.log 2024-04-23T22:00:09.365 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.47020.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82617.log 2024-04-23T22:00:09.365 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94307.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80242.log 2024-04-23T22:00:09.365 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.94307.log.gz 2024-04-23T22:00:09.365 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.82617.log: 92.0% -- replaced with /var/log/ceph/cluster2-client.mirror.67927.log.gz 2024-04-23T22:00:09.365 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52527.log 2024-04-23T22:00:09.365 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.82617.log.gz 2024-04-23T22:00:09.365 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.80242.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.103758.log 2024-04-23T22:00:09.365 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.80242.log.gz 2024-04-23T22:00:09.365 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52527.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.0.64503.log 2024-04-23T22:00:09.365 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103758.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54803.log 2024-04-23T22:00:09.365 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.0.64503.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.103758.log.gz 2024-04-23T22:00:09.366 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67056.log 2024-04-23T22:00:09.366 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57255.log/var/log/ceph/cluster2-client.mirror.54803.log: 2024-04-23T22:00:09.366 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.67056.log: gzip 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.67056.log.gz -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98433.log 2024-04-23T22:00:09.366 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.366 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57255.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44171.log 2024-04-23T22:00:09.366 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98433.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.37879.log 2024-04-23T22:00:09.366 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44171.log: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.62975.log.gz 2024-04-23T22:00:09.366 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99076.log 2024-04-23T22:00:09.366 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster2-client.mirror.37879.log: -- /var/log/ceph/cluster1-client.admin.45616.log 2024-04-23T22:00:09.366 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40682.log 2024-04-23T22:00:09.366 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99076.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58178.log 2024-04-23T22:00:09.366 INFO:teuthology.orchestra.run.smithi150.stderr: 90.6% -- replaced with /var/log/ceph/cluster2-client.mirror.98433.log.gz 2024-04-23T22:00:09.366 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.45616.log: 88.2%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40135.log 2024-04-23T22:00:09.366 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.99076.log.gz 2024-04-23T22:00:09.367 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40682.log: /var/log/ceph/cluster2-client.mirror.58178.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45102.log 2024-04-23T22:00:09.367 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.46150.log.gz 2024-04-23T22:00:09.367 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/cluster2-client.mirror.40135.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74091.log 2024-04-23T22:00:09.367 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.45102.log: 88.2% -- replaced with /var/log/ceph/cluster1-client.mirror.46217.log.gz 2024-04-23T22:00:09.367 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80775.log 2024-04-23T22:00:09.367 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74091.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47470.log 2024-04-23T22:00:09.367 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.80775.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.74091.log.gz 2024-04-23T22:00:09.367 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92915.log 2024-04-23T22:00:09.367 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.47470.log: gzip -5 --verbose 88.2% -- -- replaced with /var/log/ceph/cluster2-client.mirror.39747.log.gz 2024-04-23T22:00:09.367 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster2-client.mirror.86412.log 2024-04-23T22:00:09.367 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.92915.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51270.log 2024-04-23T22:00:09.367 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.92915.log.gz 2024-04-23T22:00:09.367 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86412.log: 92.1% -- replaced with /var/log/ceph/cluster2-client.mirror.80775.log.gz 2024-04-23T22:00:09.367 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.40426.log.gz 2024-04-23T22:00:09.368 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster2-client.mirror.93789.log 2024-04-23T22:00:09.368 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.51270.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.40523.log.gz 2024-04-23T22:00:09.368 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.86412.log.gz 2024-04-23T22:00:09.368 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster2-client.mirror.84726.log 2024-04-23T22:00:09.368 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.93789.log: gzip -5 --verbose -- 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.93789.log.gz 2024-04-23T22:00:09.368 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster2-client.mirror.106379.log 2024-04-23T22:00:09.368 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.84726.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102383.log 2024-04-23T22:00:09.368 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106379.log: gzip -5 --verbose -- 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.41361.log.gz 2024-04-23T22:00:09.368 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster1-client.mirror.100943.log 2024-04-23T22:00:09.368 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.41105.log.gz 2024-04-23T22:00:09.368 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102383.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.41396.log.gz 2024-04-23T22:00:09.368 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66932.log 2024-04-23T22:00:09.368 INFO:teuthology.orchestra.run.smithi150.stderr: 86.4% 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.84726.log.gz 2024-04-23T22:00:09.368 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.102383.log.gz 2024-04-23T22:00:09.368 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.100943.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74647.log 2024-04-23T22:00:09.368 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66932.log: 86.7%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76361.log 2024-04-23T22:00:09.369 INFO:teuthology.orchestra.run.smithi150.stderr: 91.6% -- replaced with /var/log/ceph/cluster1-client.mirror.100943.log.gz 2024-04-23T22:00:09.369 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74647.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.66932.log.gz 2024-04-23T22:00:09.369 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.106379.log.gz 2024-04-23T22:00:09.369 INFO:teuthology.orchestra.run.smithi150.stderr:gzip 88.2% -5 --verbose -- /var/log/ceph/cluster1-client.mirror.103171.log 2024-04-23T22:00:09.369 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.74647.log.gz 2024-04-23T22:00:09.369 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76361.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59696.log 2024-04-23T22:00:09.369 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.76361.log.gz 2024-04-23T22:00:09.369 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103171.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50808.log 2024-04-23T22:00:09.369 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89496.log 2024-04-23T22:00:09.369 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.103171.log.gz/var/log/ceph/cluster2-client.mirror.59696.log: 2024-04-23T22:00:09.369 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50808.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102811.log 2024-04-23T22:00:09.369 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66135.log 2024-04-23T22:00:09.369 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89496.log: /var/log/ceph/cluster1-client.mirror.102811.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.89496.log.gz 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102811.log.gz 2024-04-23T22:00:09.369 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.370 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50394.log 2024-04-23T22:00:09.370 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66135.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71850.log 2024-04-23T22:00:09.370 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.66135.log.gz 2024-04-23T22:00:09.370 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50394.log: /var/log/ceph/cluster2-client.mirror.71850.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43058.log 2024-04-23T22:00:09.370 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.58178.log.gz 2024-04-23T22:00:09.370 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.71850.log.gz 2024-04-23T22:00:09.370 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.51270.log.gz 2024-04-23T22:00:09.370 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89297.log 2024-04-23T22:00:09.370 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.43058.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.52499.log 2024-04-23T22:00:09.370 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89297.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102788.log 2024-04-23T22:00:09.370 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.89297.log.gz 2024-04-23T22:00:09.370 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.52499.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49909.log 2024-04-23T22:00:09.370 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102788.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102788.log.gz 2024-04-23T22:00:09.370 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45644.log 2024-04-23T22:00:09.371 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.80750.log 2024-04-23T22:00:09.371 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.49909.log: /var/log/ceph/cluster2-client.mirror.45644.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.3.97527.log 2024-04-23T22:00:09.371 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/cluster1-client.mirror.80750.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58115.log 2024-04-23T22:00:09.371 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.3.97527.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106239.log 2024-04-23T22:00:09.371 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster1-client.mirror.80750.log.gz 2024-04-23T22:00:09.371 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59484.log 2024-04-23T22:00:09.371 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58115.log: gzip -5 --verbose/var/log/ceph/cluster1-client.mirror.106239.log: -- /var/log/ceph/cluster2-client.mirror.54278.log 2024-04-23T22:00:09.371 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.59484.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88393.log 87.3% 2024-04-23T22:00:09.371 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.106239.log.gz 2024-04-23T22:00:09.371 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster2-client.mirror.54278.log: -- /var/log/ceph/cluster2-client.admin.107362.log 2024-04-23T22:00:09.371 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.0.51544.log 2024-04-23T22:00:09.371 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.107362.log: /var/log/ceph/cluster2-client.mirror.88393.log: gzip 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.107362.log.gz 2024-04-23T22:00:09.371 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97732.log 2024-04-23T22:00:09.371 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.88393.log.gz 2024-04-23T22:00:09.371 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster1-client.mirror.0.51544.log: -- /var/log/ceph/cluster1-client.mirror.39212.log 2024-04-23T22:00:09.372 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97732.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55538.log 2024-04-23T22:00:09.372 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.39212.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68385.log 2024-04-23T22:00:09.372 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.97732.log.gz 2024-04-23T22:00:09.372 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63252.log 2024-04-23T22:00:09.372 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68385.log: /var/log/ceph/cluster2-client.mirror.55538.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.91530.log 2024-04-23T22:00:09.372 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9%/var/log/ceph/cluster2-client.mirror.63252.log: -- replaced with /var/log/ceph/cluster2-client.mirror.68385.log.gz 2024-04-23T22:00:09.372 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102586.log 2024-04-23T22:00:09.372 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53963.log 2024-04-23T22:00:09.372 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102586.log: /var/log/ceph/cluster1-client.mirror.91530.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102586.log.gz 2024-04-23T22:00:09.372 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.91530.log.gz 2024-04-23T22:00:09.372 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79976.log 2024-04-23T22:00:09.372 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.53963.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88664.log 2024-04-23T22:00:09.372 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68758.log 2024-04-23T22:00:09.372 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88664.log: /var/log/ceph/cluster2-client.mirror.79976.log: 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.58334.log.gz 2024-04-23T22:00:09.373 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.88664.log.gz 2024-04-23T22:00:09.373 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73923.log 2024-04-23T22:00:09.373 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.68758.log: 87.9% -- replaced with /var/log/ceph/cluster1-client.mirror.68758.log.gz 2024-04-23T22:00:09.373 INFO:teuthology.orchestra.run.smithi150.stderr: 92.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57149.log.gz 2024-04-23T22:00:09.373 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101708.log 2024-04-23T22:00:09.373 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73923.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.73923.log.gz 2024-04-23T22:00:09.373 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71340.log 2024-04-23T22:00:09.373 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101708.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.101708.log.gz 2024-04-23T22:00:09.373 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose 91.8% -- replaced with /var/log/ceph/cluster1-client.mirror.3.97527.log.gz 2024-04-23T22:00:09.373 INFO:teuthology.orchestra.run.smithi150.stderr: -- /var/log/ceph/cluster2-client.mirror.59673.log 2024-04-23T22:00:09.373 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71340.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100228.log 2024-04-23T22:00:09.373 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.71340.log.gz 2024-04-23T22:00:09.373 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35591.log 2024-04-23T22:00:09.373 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.59673.log: /var/log/ceph/cluster2-client.mirror.100228.log: 88.5% -- replaced with /var/log/ceph/cluster1-client.admin.52499.log.gz 2024-04-23T22:00:09.373 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101213.log 90.6% -- replaced with /var/log/ceph/cluster2-client.mirror.100228.log.gz 2024-04-23T22:00:09.374 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.374 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82423.log 2024-04-23T22:00:09.374 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101213.log: /var/log/ceph/cluster1-client.mirror.35591.log: 86.6%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66038.log 2024-04-23T22:00:09.374 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.82423.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.82423.log.gz 2024-04-23T22:00:09.374 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71456.log -- replaced with /var/log/ceph/cluster1-client.mirror.101213.log.gz 2024-04-23T22:00:09.374 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.374 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61002.log 2024-04-23T22:00:09.374 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71456.log: /var/log/ceph/cluster2-client.mirror.66038.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.71456.log.gz 2024-04-23T22:00:09.374 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.66038.log.gz 2024-04-23T22:00:09.374 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98053.log 2024-04-23T22:00:09.374 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/cluster2-client.mirror.98053.log: 88.0% -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69046.log 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.37879.log.gz 2024-04-23T22:00:09.374 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.374 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.37813.log.gz 2024-04-23T22:00:09.374 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.61002.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.37914.log.gz 2024-04-23T22:00:09.374 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.38015.log.gz 2024-04-23T22:00:09.375 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.38187.log.gz 2024-04-23T22:00:09.375 INFO:teuthology.orchestra.run.smithi150.stderr: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.98053.log.gz 2024-04-23T22:00:09.375 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91700.log 2024-04-23T22:00:09.375 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69046.log: 87.1% -- replaced with /var/log/ceph/cluster2-client.mirror.69046.log.gz 2024-04-23T22:00:09.375 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.54803.log.gz 2024-04-23T22:00:09.375 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.54908.log.gz 2024-04-23T22:00:09.375 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104790.log 2024-04-23T22:00:09.375 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.91700.log: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.55083.log.gz 2024-04-23T22:00:09.375 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.91700.log.gz 2024-04-23T22:00:09.375 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74849.log 2024-04-23T22:00:09.375 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104790.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.45102.log.gz 2024-04-23T22:00:09.375 INFO:teuthology.orchestra.run.smithi150.stderr: 83.3% -- replaced with /var/log/ceph/cluster2-client.mirror.104790.log.gz 2024-04-23T22:00:09.388 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.40682.log.gz 2024-04-23T22:00:09.388 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster2-client.mirror.80064.log 2024-04-23T22:00:09.388 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74849.log: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.53963.log.gz 2024-04-23T22:00:09.388 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.54278.log.gz 2024-04-23T22:00:09.388 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.74849.log.gz 2024-04-23T22:00:09.388 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46350.log 2024-04-23T22:00:09.388 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.80064.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.80064.log.gz 2024-04-23T22:00:09.389 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.48475.log.gz 2024-04-23T22:00:09.389 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.49188.log.gz 2024-04-23T22:00:09.389 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.48440.log.gz 2024-04-23T22:00:09.389 INFO:teuthology.orchestra.run.smithi150.stderr: 93.6% -- replaced with /var/log/ceph/cluster1-client.mirror.0.64503.log.gz 2024-04-23T22:00:09.389 INFO:teuthology.orchestra.run.smithi150.stderr: 84.5% -- replaced with /var/log/ceph/cluster2-client.mirror.59673.log.gz 2024-04-23T22:00:09.389 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.55638.log.gz 2024-04-23T22:00:09.389 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.51463.log.gz 2024-04-23T22:00:09.389 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74148.log 2024-04-23T22:00:09.389 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46350.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42767.log 2024-04-23T22:00:09.389 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74148.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93391.log 2024-04-23T22:00:09.389 INFO:teuthology.orchestra.run.smithi150.stderr: 92.1% -- replaced with /var/log/ceph/cluster2-client.mirror.74148.log.gz 2024-04-23T22:00:09.389 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.42767.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74056.log 2024-04-23T22:00:09.389 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.93391.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63717.log 2024-04-23T22:00:09.389 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74056.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.93391.log.gz 2024-04-23T22:00:09.390 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.74056.log.gzgzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79543.log 2024-04-23T22:00:09.390 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.390 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.63717.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95617.log 2024-04-23T22:00:09.390 INFO:teuthology.orchestra.run.smithi150.stderr: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.63717.log.gz/var/log/ceph/cluster2-client.mirror.79543.log: 2024-04-23T22:00:09.390 INFO:teuthology.orchestra.run.smithi150.stderr:gzip 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.79543.log.gz 2024-04-23T22:00:09.390 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.3.103855.log 2024-04-23T22:00:09.390 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95617.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34770.log 2024-04-23T22:00:09.390 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.95617.log.gz 2024-04-23T22:00:09.390 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.3.103855.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106780.log 2024-04-23T22:00:09.390 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34770.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34770.log.gz 2024-04-23T22:00:09.390 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65728.log 2024-04-23T22:00:09.390 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106780.log: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.53227.log.gz 2024-04-23T22:00:09.390 INFO:teuthology.orchestra.run.smithi150.stderr: 91.4% -- replaced with /var/log/ceph/cluster2-client.mirror.106780.log.gz 2024-04-23T22:00:09.390 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90049.log 2024-04-23T22:00:09.391 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.65728.log: 93.1% -- replaced with /var/log/ceph/cluster1-client.mirror.50828.log.gz 2024-04-23T22:00:09.391 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.65728.log.gz 2024-04-23T22:00:09.391 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61127.log 2024-04-23T22:00:09.391 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90049.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.90049.log.gz 2024-04-23T22:00:09.391 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85330.log 2024-04-23T22:00:09.391 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2%/var/log/ceph/cluster2-client.mirror.61127.log: -- replaced with /var/log/ceph/cluster2-client.mirror.42767.log.gz 2024-04-23T22:00:09.391 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.43058.log.gz 2024-04-23T22:00:09.391 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.44171.log.gz 2024-04-23T22:00:09.393 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.40135.log.gz 2024-04-23T22:00:09.397 INFO:teuthology.orchestra.run.smithi150.stderr: 89.4% -- replaced with /var/log/ceph/cluster2-client.mirror.58115.log.gz 2024-04-23T22:00:09.397 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88333.log 2024-04-23T22:00:09.400 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85330.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.85330.log.gz 2024-04-23T22:00:09.401 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.55538.log.gz 2024-04-23T22:00:09.404 INFO:teuthology.orchestra.run.smithi150.stderr: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.35591.log.gz 2024-04-23T22:00:09.405 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% -- replaced with /var/log/ceph/cluster2-client.mirror.50808.log.gz 2024-04-23T22:00:09.418 INFO:teuthology.orchestra.run.smithi150.stderr: 93.3% -- replaced with /var/log/ceph/cluster2-client.mirror.56468.log.gz 2024-04-23T22:00:09.422 INFO:teuthology.orchestra.run.smithi150.stderr: 92.6% -- replaced with /var/log/ceph/cluster1-client.mirror.57255.log.gz 2024-04-23T22:00:09.423 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.46888.log.gz 2024-04-23T22:00:09.423 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.47020.log.gz 2024-04-23T22:00:09.424 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.47470.log.gz 2024-04-23T22:00:09.424 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42732.log 2024-04-23T22:00:09.431 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88333.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.46535.log.gz 2024-04-23T22:00:09.438 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster1-client.admin.45616.log.gz 2024-04-23T22:00:09.439 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.49909.log.gz 2024-04-23T22:00:09.442 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100328.log 2024-04-23T22:00:09.446 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.42732.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.42732.log.gz 2024-04-23T22:00:09.451 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45393.log 2024-04-23T22:00:09.458 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100328.log: 89.1% 92.7% -- replaced with /var/log/ceph/cluster1-client.mirror.44569.log.gz 2024-04-23T22:00:09.458 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.100328.log.gz 2024-04-23T22:00:09.458 INFO:teuthology.orchestra.run.smithi150.stderr: 97.2% -- replaced with /var/log/ceph/cluster2-client.mirror.56510.log.gz 2024-04-23T22:00:09.459 INFO:teuthology.orchestra.run.smithi150.stderr: 84.5% -- replaced with /var/log/ceph/cluster2-client.mirror.60535.log.gz 2024-04-23T22:00:09.460 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.63252.log.gz 2024-04-23T22:00:09.462 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.1.104298.log 2024-04-23T22:00:09.462 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.45393.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.104412.log 2024-04-23T22:00:09.462 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.45393.log.gz 2024-04-23T22:00:09.467 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.1.104298.log: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.1.104298.log.gz 2024-04-23T22:00:09.469 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99240.log 2024-04-23T22:00:09.469 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.104412.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.104412.log.gz 2024-04-23T22:00:09.469 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41008.log 2024-04-23T22:00:09.469 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99240.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.99240.log.gz 2024-04-23T22:00:09.471 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81691.log 2024-04-23T22:00:09.471 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41008.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.45513.log 2024-04-23T22:00:09.471 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81691.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.81691.log.gz 2024-04-23T22:00:09.475 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.41008.log.gz 2024-04-23T22:00:09.475 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85866.log 2024-04-23T22:00:09.475 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.45513.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86625.log 2024-04-23T22:00:09.475 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85866.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.85866.log.gz 2024-04-23T22:00:09.475 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.50394.log.gz 2024-04-23T22:00:09.475 INFO:teuthology.orchestra.run.smithi150.stderr: -- /var/log/ceph/cluster1-client.mirror.1.35486.log 2024-04-23T22:00:09.475 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86625.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.86625.log.gz 2024-04-23T22:00:09.475 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75274.log 2024-04-23T22:00:09.475 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.1.35486.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72354.log 2024-04-23T22:00:09.477 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75274.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.75274.log.gz 2024-04-23T22:00:09.477 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.91575.log 2024-04-23T22:00:09.478 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.72354.log: 93.1% -- replaced with /var/log/ceph/cluster2-client.mirror.57685.log.gz 2024-04-23T22:00:09.479 INFO:teuthology.orchestra.run.smithi150.stderr: 91.9% -- replaced with /var/log/ceph/cluster2-client.mirror.72354.log.gz 2024-04-23T22:00:09.480 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87244.log 2024-04-23T22:00:09.481 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.91575.log: 91.7% -- replaced with /var/log/ceph/cluster1-client.mirror.91575.log.gz 2024-04-23T22:00:09.502 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106760.log 2024-04-23T22:00:09.502 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87244.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.87244.log.gz 2024-04-23T22:00:09.504 INFO:teuthology.orchestra.run.smithi148.stderr: 93.1% -- replaced with /var/log/ceph/cluster1.audit.log.gz 2024-04-23T22:00:09.520 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69913.log 2024-04-23T22:00:09.520 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106760.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.106760.log.gz 2024-04-23T22:00:09.530 INFO:teuthology.orchestra.run.smithi150.stderr: 92.6% -- replaced with /var/log/ceph/cluster2-client.mirror.3.45689.log.gz 2024-04-23T22:00:09.536 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.39167.log 2024-04-23T22:00:09.537 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.69913.log: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.69913.log.gz 2024-04-23T22:00:09.540 INFO:teuthology.orchestra.run.smithi150.stderr: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.58029.log.gz 2024-04-23T22:00:09.547 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.36312.log 2024-04-23T22:00:09.559 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.39167.log: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.39167.log.gz 2024-04-23T22:00:09.585 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.64037.log.gz 2024-04-23T22:00:09.585 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65610.log 2024-04-23T22:00:09.586 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.36312.log: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.58072.log.gz 2024-04-23T22:00:09.597 INFO:teuthology.orchestra.run.smithi150.stderr: 95.5%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106739.log 2024-04-23T22:00:09.597 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.65610.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.65610.log.gz 2024-04-23T22:00:09.600 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.88333.log.gz 2024-04-23T22:00:09.601 INFO:teuthology.orchestra.run.smithi150.stderr: 95.4% -- replaced with /var/log/ceph/cluster2-client.mirror.61002.log.gz 2024-04-23T22:00:09.602 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100705.log 2024-04-23T22:00:09.602 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106739.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.106739.log.gz 2024-04-23T22:00:09.604 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.39100.log 2024-04-23T22:00:09.604 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100705.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.100705.log.gz 2024-04-23T22:00:09.605 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40197.log 2024-04-23T22:00:09.605 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.39100.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49103.log 2024-04-23T22:00:09.607 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40197.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88946.log 2024-04-23T22:00:09.607 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.49103.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86706.log 2024-04-23T22:00:09.608 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.88946.log: 87.8% 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.40197.log.gz -- replaced with /var/log/ceph/cluster2-client.mirror.39100.log.gz 2024-04-23T22:00:09.608 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.609 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.49103.log.gz 2024-04-23T22:00:09.611 INFO:teuthology.orchestra.run.smithi150.stderr: 91.8% -- replaced with /var/log/ceph/cluster1-client.mirror.88946.log.gz 2024-04-23T22:00:09.611 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103810.log 2024-04-23T22:00:09.612 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86706.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.86706.log.gz 2024-04-23T22:00:09.612 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62797.log 2024-04-23T22:00:09.614 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.103810.log: 91.2% -- replaced with /var/log/ceph/cluster2-client.mirror.103810.log.gz 2024-04-23T22:00:09.614 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.31102.log 2024-04-23T22:00:09.630 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.62797.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94779.log 2024-04-23T22:00:09.636 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105799.log 2024-04-23T22:00:09.637 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94779.log: /var/log/ceph/cluster2-client.admin.31102.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.31102.log.gz 2024-04-23T22:00:09.647 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55013.log 2024-04-23T22:00:09.647 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.94779.log.gz 2024-04-23T22:00:09.648 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105799.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.105799.log.gz 2024-04-23T22:00:09.660 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50588.log 2024-04-23T22:00:09.664 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55013.log: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.55013.log.gz 2024-04-23T22:00:09.672 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.79976.log.gz 2024-04-23T22:00:09.674 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95921.log 2024-04-23T22:00:09.679 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50588.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81539.log 2024-04-23T22:00:09.679 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95921.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.95921.log.gz 2024-04-23T22:00:09.691 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101033.log 2024-04-23T22:00:09.691 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81539.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.81539.log.gz 2024-04-23T22:00:09.691 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- 94.1% -- replaced with /var/log/ceph/cluster2-client.mirror.1.60061.log.gz 2024-04-23T22:00:09.691 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster1-client.mirror.57298.log 2024-04-23T22:00:09.691 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101033.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90906.log 2024-04-23T22:00:09.691 INFO:teuthology.orchestra.run.smithi150.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.101033.log.gz 2024-04-23T22:00:09.691 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57298.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76082.log 2024-04-23T22:00:09.691 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90906.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76124.log 2024-04-23T22:00:09.691 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.90906.log.gz 2024-04-23T22:00:09.691 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76082.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90440.log 2024-04-23T22:00:09.691 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76124.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.76082.log.gz 2024-04-23T22:00:09.691 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.76124.log.gz 2024-04-23T22:00:09.691 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.63839.log.gzgzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63360.log 2024-04-23T22:00:09.692 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.692 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90440.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67261.log 2024-04-23T22:00:09.692 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.90440.log.gz 2024-04-23T22:00:09.692 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.63360.log: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.63360.log.gz 2024-04-23T22:00:09.692 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.39712.log 2024-04-23T22:00:09.692 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.67261.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52772.log 2024-04-23T22:00:09.692 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.39712.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98484.log 2024-04-23T22:00:09.692 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52772.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35968.log 2024-04-23T22:00:09.692 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster2-client.mirror.98484.log: -- /var/log/ceph/cluster1-client.mirror.0.44654.log 2024-04-23T22:00:09.692 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.35968.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.98484.log.gz 2024-04-23T22:00:09.692 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41652.log 2024-04-23T22:00:09.692 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.0.44654.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51781.log 2024-04-23T22:00:09.692 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41652.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54978.log 2024-04-23T22:00:09.693 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.51781.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75316.log 2024-04-23T22:00:09.693 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.54978.log: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.52772.log.gz 2024-04-23T22:00:09.694 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83072.log 2024-04-23T22:00:09.694 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.54978.log.gz 2024-04-23T22:00:09.694 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75316.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.75316.log.gz 2024-04-23T22:00:09.696 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60150.log/var/log/ceph/cluster2-client.mirror.83072.log: 2024-04-23T22:00:09.709 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.30969.log 2024-04-23T22:00:09.713 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60150.log: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.60150.log.gz 2024-04-23T22:00:09.715 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98604.log 2024-04-23T22:00:09.715 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.30969.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.30969.log.gz 2024-04-23T22:00:09.715 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105037.log 2024-04-23T22:00:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98604.log: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.98604.log.gz 2024-04-23T22:00:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58502.log 2024-04-23T22:00:09.720 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.105037.log: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.105037.log.gz 2024-04-23T22:00:09.720 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102631.log 2024-04-23T22:00:09.720 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58502.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104119.log 2024-04-23T22:00:09.720 INFO:teuthology.orchestra.run.smithi150.stderr: 93.4%/var/log/ceph/cluster1-client.mirror.102631.log: -- replaced with /var/log/ceph/cluster1-client.mirror.0.44654.log.gz 2024-04-23T22:00:09.720 INFO:teuthology.orchestra.run.smithi150.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.102631.log.gz 2024-04-23T22:00:09.720 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57662.log 2024-04-23T22:00:09.721 INFO:teuthology.orchestra.run.smithi150.stderr: 80.1% -- replaced with /var/log/ceph/cluster1-client.admin.35968.log.gz 2024-04-23T22:00:09.722 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104119.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.39712.log.gz 2024-04-23T22:00:09.723 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.58502.log.gz 2024-04-23T22:00:09.731 INFO:teuthology.orchestra.run.smithi150.stderr: 92.7% -- replaced with /var/log/ceph/cluster1-client.mirror.51781.log.gz 2024-04-23T22:00:09.731 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44449.log 2024-04-23T22:00:09.734 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57662.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.41652.log.gz 2024-04-23T22:00:09.737 INFO:teuthology.orchestra.run.smithi150.stderr: 92.7% -- replaced with /var/log/ceph/cluster1-client.mirror.39212.log.gz 2024-04-23T22:00:09.745 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60979.log 2024-04-23T22:00:09.747 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44449.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.44449.log.gz 2024-04-23T22:00:09.756 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster1-client.mirror.49417.log.gz 2024-04-23T22:00:09.756 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.50491.log.gz 2024-04-23T22:00:09.757 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.50588.log.gz 2024-04-23T22:00:09.757 INFO:teuthology.orchestra.run.smithi150.stderr: 95.5% -- replaced with /var/log/ceph/cluster2-client.mirror.105847.log.gz 2024-04-23T22:00:09.757 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.56445.log 2024-04-23T22:00:09.757 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60979.log: 84.3% -- replaced with /var/log/ceph/cluster2-client.mirror.60979.log.gz 2024-04-23T22:00:09.757 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67835.log 2024-04-23T22:00:09.757 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.56445.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87143.log 2024-04-23T22:00:09.757 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.67835.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.67835.log.gz 2024-04-23T22:00:09.757 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79247.log 2024-04-23T22:00:09.757 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87143.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.87143.log.gz 2024-04-23T22:00:09.757 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75970.log 2024-04-23T22:00:09.757 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.79247.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.79247.log.gz 2024-04-23T22:00:09.757 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50103.log 2024-04-23T22:00:09.757 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75970.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.75970.log.gz 2024-04-23T22:00:09.757 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97527.log 2024-04-23T22:00:09.757 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50103.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47893.log 2024-04-23T22:00:09.758 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97527.log: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.50103.log.gz 2024-04-23T22:00:09.759 INFO:teuthology.orchestra.run.smithi150.stderr: 91.9% -- replaced with /var/log/ceph/cluster2-client.mirror.97527.log.gz 2024-04-23T22:00:09.759 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60462.log 2024-04-23T22:00:09.762 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.47893.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45845.log 2024-04-23T22:00:09.764 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60462.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86234.log 2024-04-23T22:00:09.764 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.45845.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61104.log 2024-04-23T22:00:09.764 INFO:teuthology.orchestra.run.smithi150.stderr: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.45845.log.gz 2024-04-23T22:00:09.767 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.86234.log: 91.5% -- replaced with /var/log/ceph/cluster1-client.mirror.86234.log.gz 2024-04-23T22:00:09.767 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67219.log 2024-04-23T22:00:09.767 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.61104.log: 84.5% -- replaced with /var/log/ceph/cluster2-client.mirror.61104.log.gz 2024-04-23T22:00:09.768 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87209.log 2024-04-23T22:00:09.768 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1%/var/log/ceph/cluster2-client.mirror.67219.log: -- replaced with /var/log/ceph/cluster2-client.mirror.60671.log.gz 2024-04-23T22:00:09.770 INFO:teuthology.orchestra.run.smithi150.stderr: 92.3% -- replaced with /var/log/ceph/cluster2-client.mirror.67219.log.gz 93.0% -- replaced with /var/log/ceph/cluster1-client.mirror.57662.log.gz 2024-04-23T22:00:09.771 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.771 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98184.log 2024-04-23T22:00:09.773 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87209.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.87209.log.gz 2024-04-23T22:00:09.773 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82037.log 2024-04-23T22:00:09.773 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5/var/log/ceph/cluster2-client.mirror.98184.log: --verbose -- /var/log/ceph/cluster1-client.mirror.94055.log 2024-04-23T22:00:09.773 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.82037.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.82037.log.gz 2024-04-23T22:00:09.773 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.98184.log.gz 2024-04-23T22:00:09.775 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.47893.log.gz 2024-04-23T22:00:09.776 INFO:teuthology.orchestra.run.smithi150.stderr: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.56445.log.gz 2024-04-23T22:00:09.776 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67447.log 2024-04-23T22:00:09.777 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105697.log 2024-04-23T22:00:09.777 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.94055.log: /var/log/ceph/cluster2-client.mirror.67447.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.67447.log.gz 2024-04-23T22:00:09.778 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.31182.log 2024-04-23T22:00:09.778 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.105697.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.105697.log.gz 2024-04-23T22:00:09.851 INFO:teuthology.orchestra.run.smithi150.stderr: 92.1%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104071.log 2024-04-23T22:00:09.852 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.94055.log.gz 2024-04-23T22:00:09.852 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.31182.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90235.log 2024-04-23T22:00:09.852 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104071.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.31182.log.gz 2024-04-23T22:00:09.852 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97845.log 2024-04-23T22:00:09.852 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90235.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.90235.log.gz 2024-04-23T22:00:09.852 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster1-client.mirror.3.103855.log.gz 2024-04-23T22:00:09.852 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84974.log 2024-04-23T22:00:09.852 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97845.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85500.log 2024-04-23T22:00:09.852 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.84974.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.84974.log.gz 2024-04-23T22:00:09.852 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94226.log 2024-04-23T22:00:09.852 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85500.log: 90.6% -- replaced with /var/log/ceph/cluster2-client.mirror.97845.log.gz 2024-04-23T22:00:09.852 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.85500.log.gzgzip -5 --verbose 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.104071.log.gz 2024-04-23T22:00:09.853 INFO:teuthology.orchestra.run.smithi150.stderr: -- /var/log/ceph/cluster2-client.mirror.0.37245.log 2024-04-23T22:00:09.853 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.853 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94226.log: gzip -5 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.94226.log.gz 2024-04-23T22:00:09.853 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/cluster2-client.mirror.100541.log 2024-04-23T22:00:09.853 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79279.log 2024-04-23T22:00:09.853 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.37510.log 2024-04-23T22:00:09.853 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104654.log 2024-04-23T22:00:09.853 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.107446.log 2024-04-23T22:00:09.853 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100683.log 2024-04-23T22:00:09.853 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.0.37245.log: /var/log/ceph/cluster2-client.mirror.100541.log: /var/log/ceph/cluster1-client.mirror.79279.log: /var/log/ceph/cluster2-client.mirror.37510.log: /var/log/ceph/cluster2-client.mirror.104654.log: 88.2%gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44754.log 2024-04-23T22:00:09.853 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.100541.log.gz 2024-04-23T22:00:09.853 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100683.log: -- replaced with /var/log/ceph/cluster1-client.mirror.79279.log.gz 2024-04-23T22:00:09.853 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.107446.log: 91.2% -- replaced with /var/log/ceph/cluster2-client.mirror.104654.log.gz 2024-04-23T22:00:09.853 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.107446.log.gz 2024-04-23T22:00:09.853 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87329.log 2024-04-23T22:00:09.854 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.44754.log: gzip -5 --verbose -- 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.100683.log.gz 2024-04-23T22:00:09.854 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster2-client.mirror.56967.log 2024-04-23T22:00:09.854 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87329.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64287.log 2024-04-23T22:00:09.854 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.87329.log.gz 2024-04-23T22:00:09.854 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56967.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104728.log 2024-04-23T22:00:09.854 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.64287.log: 1.3%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48343.log 2024-04-23T22:00:09.854 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.64287.log.gz 2024-04-23T22:00:09.854 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104728.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66410.log 2024-04-23T22:00:09.854 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48343.log: 88.2%gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.56488.log 2024-04-23T22:00:09.854 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.104728.log.gz 2024-04-23T22:00:09.854 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.66410.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.37510.log.gz 87.9% -- replaced with /var/log/ceph/cluster1-client.mirror.66410.log.gz 2024-04-23T22:00:09.854 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% 2024-04-23T22:00:09.854 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.44754.log.gz 2024-04-23T22:00:09.854 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93215.log 2024-04-23T22:00:09.855 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/cluster1-client.mirror.56488.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68783.log 2024-04-23T22:00:09.855 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.93215.log: gzip -5 --verbose -- 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.93215.log.gz 2024-04-23T22:00:09.855 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster2-client.mirror.95078.log 2024-04-23T22:00:09.855 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99493.log 2024-04-23T22:00:09.855 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95078.log: 87.9% 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.48343.log.gz 2024-04-23T22:00:09.855 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.95078.log.gz 2024-04-23T22:00:09.855 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8%gzip -5/var/log/ceph/cluster2-client.mirror.99493.log: --verbose -- /var/log/ceph/cluster2-client.mirror.44368.log 2024-04-23T22:00:09.855 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.56967.log.gz 2024-04-23T22:00:09.855 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43543.log 2024-04-23T22:00:09.855 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68783.log: /var/log/ceph/cluster2-client.mirror.44368.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58767.log 2024-04-23T22:00:09.855 INFO:teuthology.orchestra.run.smithi150.stderr: 90.6%/var/log/ceph/cluster2-client.mirror.43543.log: -- replaced with /var/log/ceph/cluster2-client.mirror.99493.log.gz 2024-04-23T22:00:09.855 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48925.log 2024-04-23T22:00:09.855 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57857.log 2024-04-23T22:00:09.855 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58767.log: 70.3% -- replaced with /var/log/ceph/cluster2-client.admin.36312.log.gz 2024-04-23T22:00:09.856 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48925.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59752.log 2024-04-23T22:00:09.856 INFO:teuthology.orchestra.run.smithi150.stderr: 92.0% -- replaced with /var/log/ceph/cluster2-client.mirror.68783.log.gz 2024-04-23T22:00:09.856 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.31142.log 2024-04-23T22:00:09.856 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57857.log: /var/log/ceph/cluster2-client.mirror.59752.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52568.log 2024-04-23T22:00:09.856 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.44368.log.gz 2024-04-23T22:00:09.856 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.102676.log 2024-04-23T22:00:09.856 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52568.log: /var/log/ceph/cluster2-client.admin.31142.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71537.log 2024-04-23T22:00:09.856 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.31142.log.gz 2024-04-23T22:00:09.856 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102676.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100421.log 2024-04-23T22:00:09.856 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71537.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.43543.log.gz 2024-04-23T22:00:09.856 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% 86.7% -- replaced with /var/log/ceph/cluster2-client.mirror.71537.log.gz -- replaced with /var/log/ceph/cluster1-client.mirror.102676.log.gz 2024-04-23T22:00:09.857 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.857 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47408.log 2024-04-23T22:00:09.857 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100421.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.100421.log.gz 2024-04-23T22:00:09.857 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.47408.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.48925.log.gz 2024-04-23T22:00:09.857 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100350.log 2024-04-23T22:00:09.857 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67346.log 2024-04-23T22:00:09.857 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100350.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97754.log 2024-04-23T22:00:09.857 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.100350.log.gz 2024-04-23T22:00:09.857 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.67346.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70331.log 2024-04-23T22:00:09.857 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.67346.log.gz 2024-04-23T22:00:09.858 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97754.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.97754.log.gz 2024-04-23T22:00:09.858 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.47408.log.gz 2024-04-23T22:00:09.858 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66378.log 2024-04-23T22:00:09.858 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.70331.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81262.log 2024-04-23T22:00:09.858 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66378.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.35277.log 2024-04-23T22:00:09.858 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.70331.log.gz 2024-04-23T22:00:09.858 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9%/var/log/ceph/cluster2-client.mirror.81262.log: -- replaced with /var/log/ceph/cluster2-client.mirror.66378.log.gz 2024-04-23T22:00:09.858 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64014.log 2024-04-23T22:00:09.858 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.35277.log: 0.0%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58439.log 2024-04-23T22:00:09.858 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.admin.35277.log.gz 2024-04-23T22:00:09.858 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.81262.log.gz 2024-04-23T22:00:09.858 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.64014.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82884.log 2024-04-23T22:00:09.858 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58439.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.103195.log 2024-04-23T22:00:09.859 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49138.log 2024-04-23T22:00:09.859 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103195.log: /var/log/ceph/cluster2-client.mirror.82884.log: 54.0% -- replaced with /var/log/ceph/cluster1-client.mirror.103195.log.gz 2024-04-23T22:00:09.859 INFO:teuthology.orchestra.run.smithi150.stderr: 84.5% 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.82884.log.gz 2024-04-23T22:00:09.859 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.64014.log.gz 2024-04-23T22:00:09.859 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98462.log 2024-04-23T22:00:09.859 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60896.log 2024-04-23T22:00:09.859 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98462.log: /var/log/ceph/cluster2-client.mirror.49138.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96807.log 2024-04-23T22:00:09.859 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69450.log 2024-04-23T22:00:09.859 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60896.log: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.58439.log.gz 2024-04-23T22:00:09.859 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.96807.log: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.98462.log.gz 2024-04-23T22:00:09.859 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.96807.log.gz 2024-04-23T22:00:09.859 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94164.log 2024-04-23T22:00:09.859 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80508.log 2024-04-23T22:00:09.860 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94164.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.94164.log.gz 2024-04-23T22:00:09.860 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69450.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.69450.log.gz 88.7% 2024-04-23T22:00:09.860 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.49138.log.gz 2024-04-23T22:00:09.860 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.78122.log 2024-04-23T22:00:09.860 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.80508.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40100.log 2024-04-23T22:00:09.860 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9%/var/log/ceph/cluster2-client.mirror.78122.log: -- replaced with /var/log/ceph/cluster2-client.mirror.80508.log.gz 2024-04-23T22:00:09.860 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.2.35508.log 2024-04-23T22:00:09.860 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.78122.log.gz 2024-04-23T22:00:09.860 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80861.log 2024-04-23T22:00:09.860 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40100.log: /var/log/ceph/cluster1-client.mirror.2.35508.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92218.log 2024-04-23T22:00:09.860 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.80861.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84870.log 2024-04-23T22:00:09.860 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.92218.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.92218.log.gz 2024-04-23T22:00:09.860 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101056.log 2024-04-23T22:00:09.861 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.40100.log.gz 2024-04-23T22:00:09.861 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.84870.log: 92.1% -- replaced with /var/log/ceph/cluster2-client.mirror.84870.log.gz 2024-04-23T22:00:09.861 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58847.log 2024-04-23T22:00:09.861 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101056.log: 86.8% -- replaced with /var/log/ceph/cluster1-client.mirror.101056.log.gz 2024-04-23T22:00:09.861 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52912.log 2024-04-23T22:00:09.861 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58847.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91103.log 2024-04-23T22:00:09.861 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.80861.log.gz 2024-04-23T22:00:09.861 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52912.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86833.log 2024-04-23T22:00:09.861 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.91103.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54558.log 2024-04-23T22:00:09.861 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.91103.log.gz 2024-04-23T22:00:09.861 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86833.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.86833.log.gz 2024-04-23T22:00:09.861 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59906.log 2024-04-23T22:00:09.862 INFO:teuthology.orchestra.run.smithi150.stderr: 84.5% -- replaced with /var/log/ceph/cluster2-client.mirror.58847.log.gz 2024-04-23T22:00:09.862 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.54558.log: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.52912.log.gz 2024-04-23T22:00:09.862 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.54558.log.gz 2024-04-23T22:00:09.862 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69224.log 2024-04-23T22:00:09.862 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.59906.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52381.log 2024-04-23T22:00:09.862 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69224.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.69224.log.gz 2024-04-23T22:00:09.862 INFO:teuthology.orchestra.run.smithi150.stderr: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.59906.log.gz 2024-04-23T22:00:09.862 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99011.log 2024-04-23T22:00:09.862 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81031.log 2024-04-23T22:00:09.862 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52381.log: 95.5% -- replaced with /var/log/ceph/cluster2-client.mirror.62797.log.gz 2024-04-23T22:00:09.862 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83898.log 2024-04-23T22:00:09.862 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99011.log: /var/log/ceph/cluster2-client.mirror.81031.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.81031.log.gz 2024-04-23T22:00:09.862 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.99011.log.gz 2024-04-23T22:00:09.862 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47602.log 2024-04-23T22:00:09.863 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.83898.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106655.log 2024-04-23T22:00:09.863 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9%/var/log/ceph/cluster2-client.mirror.47602.log: -- replaced with /var/log/ceph/cluster2-client.mirror.83898.log.gz 2024-04-23T22:00:09.863 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84200.log 2024-04-23T22:00:09.863 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48149.log 2024-04-23T22:00:09.863 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.84200.log: /var/log/ceph/cluster2-client.mirror.106655.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.84200.log.gz 2024-04-23T22:00:09.863 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66791.log 2024-04-23T22:00:09.863 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48149.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.1.60671.log 2024-04-23T22:00:09.863 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5/var/log/ceph/cluster2-client.mirror.66791.log: --verbose -- /var/log/ceph/cluster2-client.mirror.56005.log 2024-04-23T22:00:09.863 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.66791.log.gz 2024-04-23T22:00:09.863 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.1.60671.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.2.57401.log 2024-04-23T22:00:09.863 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56005.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101281.log 2024-04-23T22:00:09.863 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.2.57401.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105198.log 2024-04-23T22:00:09.863 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101281.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101281.log.gz 2024-04-23T22:00:09.863 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55900.log 2024-04-23T22:00:09.864 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105198.log: 83.2% 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.105198.log.gz 2024-04-23T22:00:09.864 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.48149.log.gz 2024-04-23T22:00:09.864 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.51712.log 2024-04-23T22:00:09.864 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55900.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71421.log 2024-04-23T22:00:09.864 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster1-client.admin.51712.log: -- /var/log/ceph/cluster1-client.admin.35067.log 2024-04-23T22:00:09.864 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71421.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.71421.log.gz 2024-04-23T22:00:09.864 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98626.log 2024-04-23T22:00:09.864 INFO:teuthology.orchestra.run.smithi150.stderr: 84.6% -- replaced with /var/log/ceph/cluster2-client.mirror.58767.log.gz 2024-04-23T22:00:09.864 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.35067.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35067.log.gz 2024-04-23T22:00:09.864 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68941.log 2024-04-23T22:00:09.864 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5/var/log/ceph/cluster2-client.mirror.98626.log: --verbose -- /var/log/ceph/cluster2-client.mirror.47955.log 2024-04-23T22:00:09.864 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.98626.log.gz 2024-04-23T22:00:09.864 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68941.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.68941.log.gz 2024-04-23T22:00:09.864 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-mon.a.log 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.59752.log.gz 2024-04-23T22:00:09.864 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.864 INFO:teuthology.orchestra.run.smithi150.stderr: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.60462.log.gz 2024-04-23T22:00:09.865 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5/var/log/ceph/cluster2-client.mirror.47955.log: --verbose -- /var/log/ceph/cluster2-client.mirror.56767.log 2024-04-23T22:00:09.865 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster2-client.mirror.56767.log: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.56005.log.gz 2024-04-23T22:00:09.865 INFO:teuthology.orchestra.run.smithi150.stderr: -- /var/log/ceph/cluster2-client.mirror.35508.log 2024-04-23T22:00:09.865 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-mon.a.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46185.log 2024-04-23T22:00:09.865 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82849.log 2024-04-23T22:00:09.865 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.35508.log: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.55900.log.gz 2024-04-23T22:00:09.865 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.47955.log.gz 2024-04-23T22:00:09.865 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46185.log: 88.5% -- replaced with /var/log/ceph/cluster1-client.admin.51712.log.gz 2024-04-23T22:00:09.865 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41749.log 2024-04-23T22:00:09.865 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.82849.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.82849.log.gz 2024-04-23T22:00:09.865 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59170.log 2024-04-23T22:00:09.865 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41749.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44090.log 2024-04-23T22:00:09.865 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.59170.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69348.log 2024-04-23T22:00:09.865 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44090.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57533.log 2024-04-23T22:00:09.865 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69348.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.69348.log.gz 2024-04-23T22:00:09.870 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57456.log 2024-04-23T22:00:09.870 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57533.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.41749.log.gz 2024-04-23T22:00:09.870 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.44090.log.gz 2024-04-23T22:00:09.870 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53332.log 2024-04-23T22:00:09.870 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57456.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40973.log 2024-04-23T22:00:09.870 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.53332.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.2.106511.log 2024-04-23T22:00:09.870 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40973.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.46185.log.gz 2024-04-23T22:00:09.870 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65148.log 2024-04-23T22:00:09.871 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.2.106511.log: 92.7% -- replaced with /var/log/ceph/cluster1-client.mirror.57298.log.gz 2024-04-23T22:00:09.871 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.40973.log.gz 2024-04-23T22:00:09.872 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.61127.log.gz 2024-04-23T22:00:09.872 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74958.log 2024-04-23T22:00:09.873 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.65148.log: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.83072.log.gz 2024-04-23T22:00:09.874 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87512.log 2024-04-23T22:00:09.874 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.74958.log: 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.74958.log.gz 2024-04-23T22:00:09.879 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75239.log 2024-04-23T22:00:09.881 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87512.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.87512.log.gz 2024-04-23T22:00:09.881 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47117.log 2024-04-23T22:00:09.882 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75239.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.75239.log.gz 2024-04-23T22:00:09.882 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106019.log 2024-04-23T22:00:09.884 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.47117.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57401.log 2024-04-23T22:00:09.884 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.106019.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.106019.log.gz 2024-04-23T22:00:09.884 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53823.log 2024-04-23T22:00:09.886 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57401.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60558.log 2024-04-23T22:00:09.886 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.53823.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72775.log 2024-04-23T22:00:09.888 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60558.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77238.log 2024-04-23T22:00:09.888 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.72775.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.72775.log.gz 2024-04-23T22:00:09.888 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88579.log 2024-04-23T22:00:09.892 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77238.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.77238.log.gz 2024-04-23T22:00:09.892 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81374.log 2024-04-23T22:00:09.892 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88579.log: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.67261.log.gz 2024-04-23T22:00:09.892 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.88579.log.gz 2024-04-23T22:00:09.892 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47505.log 2024-04-23T22:00:09.892 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.107740.log 2024-04-23T22:00:09.892 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.47505.log: /var/log/ceph/cluster1-client.mirror.81374.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.47117.log.gz 2024-04-23T22:00:09.892 INFO:teuthology.orchestra.run.smithi150.stderr: 95.1% -- replaced with /var/log/ceph/cluster2-client.mirror.106655.log.gz 2024-04-23T22:00:09.902 INFO:teuthology.orchestra.run.smithi150.stderr: 91.5% -- replaced with /var/log/ceph/cluster1-client.mirror.81374.log.gz 2024-04-23T22:00:09.902 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.103486.log 2024-04-23T22:00:09.902 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.107740.log: gzip 0.0% -5 --verbose -- /var/log/ceph/cluster2-osd.2.log 2024-04-23T22:00:09.902 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.admin.107740.log.gz 2024-04-23T22:00:09.902 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103486.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57619.log 2024-04-23T22:00:09.902 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-osd.2.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.103486.log.gz 2024-04-23T22:00:09.902 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66545.log 2024-04-23T22:00:09.902 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57619.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44654.log 2024-04-23T22:00:09.903 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66545.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51941.log 2024-04-23T22:00:09.903 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44654.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54698.log 2024-04-23T22:00:09.903 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56913.log 2024-04-23T22:00:09.903 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.54698.log: /var/log/ceph/cluster2-client.mirror.51941.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.45585.log 2024-04-23T22:00:09.903 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56913.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.66545.log.gz 2024-04-23T22:00:09.903 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61414.log 2024-04-23T22:00:09.903 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.45585.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104749.log 2024-04-23T22:00:09.903 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.61414.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67194.log 2024-04-23T22:00:09.903 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104749.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.104749.log.gz 2024-04-23T22:00:09.903 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.54698.log.gz 2024-04-23T22:00:09.903 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.56913.log.gz 2024-04-23T22:00:09.903 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104454.log 2024-04-23T22:00:09.904 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.67194.log: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.67194.log.gz 2024-04-23T22:00:09.904 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/cluster1-client.admin.45585.log.gz 2024-04-23T22:00:09.905 INFO:teuthology.orchestra.run.smithi150.stderr: 73.9% -- replaced with /var/log/ceph/cluster2-client.mirror.44654.log.gz 2024-04-23T22:00:09.906 INFO:teuthology.orchestra.run.smithi150.stderr: 94.5% -- replaced with /var/log/ceph/cluster1-client.mirror.57401.log.gz 2024-04-23T22:00:09.906 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105466.log 2024-04-23T22:00:09.907 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104454.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.104454.log.gz 2024-04-23T22:00:09.908 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.47505.log.gz 2024-04-23T22:00:09.908 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.47602.log.gz 2024-04-23T22:00:09.910 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.53332.log.gz 2024-04-23T22:00:09.911 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.53823.log.gz 2024-04-23T22:00:09.915 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38591.log 2024-04-23T22:00:09.917 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105466.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48731.log 2024-04-23T22:00:09.917 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.38591.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74296.log 2024-04-23T22:00:09.918 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48731.log: 91.5% -- replaced with /var/log/ceph/cluster2-client.mirror.105466.log.gz 2024-04-23T22:00:09.918 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73331.log 2024-04-23T22:00:09.920 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74296.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75860.log 2024-04-23T22:00:09.921 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.73331.log: 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.73331.log.gz 2024-04-23T22:00:09.921 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68674.log 2024-04-23T22:00:09.921 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.38591.log.gz 2024-04-23T22:00:09.923 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75860.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.75860.log.gz 2024-04-23T22:00:09.923 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86028.log 2024-04-23T22:00:09.923 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.48731.log.gz 2024-04-23T22:00:09.926 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68674.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.68674.log.gz 2024-04-23T22:00:09.926 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66262.log 2024-04-23T22:00:09.926 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86028.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.86028.log.gz 2024-04-23T22:00:09.926 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43834.log 2024-04-23T22:00:09.926 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66262.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.66262.log.gz 2024-04-23T22:00:09.926 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79954.log 2024-04-23T22:00:09.928 INFO:teuthology.orchestra.run.smithi150.stderr: 97.1% -- replaced with /var/log/ceph/cluster2-client.mirror.74296.log.gz 2024-04-23T22:00:09.929 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74983.log 2024-04-23T22:00:09.931 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.79954.log: 91.8% -- replaced with /var/log/ceph/cluster1-client.mirror.79954.log.gz 2024-04-23T22:00:09.941 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.43834.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90490.log 2024-04-23T22:00:09.941 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74983.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50041.log 2024-04-23T22:00:09.943 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90490.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.90490.log.gz 91.8% -- replaced with /var/log/ceph/cluster2-client.mirror.74983.log.gz 2024-04-23T22:00:09.943 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.946 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.43834.log.gz 2024-04-23T22:00:09.948 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50041.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104912.log 2024-04-23T22:00:09.948 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.36121.log 2024-04-23T22:00:09.948 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104912.log: 80.3% -- replaced with /var/log/ceph/cluster2-client.mirror.104912.log.gz 2024-04-23T22:00:09.950 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92396.log 2024-04-23T22:00:09.950 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5/var/log/ceph/cluster2-client.admin.36121.log: --verbose -- /var/log/ceph/cluster2-client.mirror.54663.log 0.0% 2024-04-23T22:00:09.950 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.admin.36121.log.gz 2024-04-23T22:00:09.950 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.92396.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.92396.log.gz 2024-04-23T22:00:09.958 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105528.log 2024-04-23T22:00:09.961 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.54663.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71375.log 2024-04-23T22:00:09.961 INFO:teuthology.orchestra.run.smithi150.stderr: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.50041.log.gz 2024-04-23T22:00:09.961 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105528.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.105528.log.gz 2024-04-23T22:00:09.961 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94035.log 2024-04-23T22:00:09.976 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71375.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.71375.log.gz 2024-04-23T22:00:09.976 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42829.log 2024-04-23T22:00:09.976 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.54663.log.gz 2024-04-23T22:00:09.976 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94035.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.104298.log 2024-04-23T22:00:09.977 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.42829.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81329.log 2024-04-23T22:00:09.977 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.104298.log: 83.4% -- replaced with /var/log/ceph/cluster1-client.mirror.104298.log.gz 2024-04-23T22:00:09.977 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56183.log 2024-04-23T22:00:09.977 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.81329.log: 92.2% 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.94035.log.gz 2024-04-23T22:00:09.977 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.81329.log.gz 2024-04-23T22:00:09.977 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69081.log 2024-04-23T22:00:09.977 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56183.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88834.log 2024-04-23T22:00:09.977 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69081.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102743.log 2024-04-23T22:00:09.977 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.51941.log.gz 2024-04-23T22:00:09.977 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.69081.log.gz 2024-04-23T22:00:09.977 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88834.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70963.log 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.88834.log.gz 2024-04-23T22:00:09.977 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:09.977 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62464.log 2024-04-23T22:00:09.977 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.70963.log: /var/log/ceph/cluster1-client.mirror.102743.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.102743.log.gz 2024-04-23T22:00:09.978 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.70963.log.gz 2024-04-23T22:00:09.978 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53613.log 2024-04-23T22:00:09.978 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50669.log 2024-04-23T22:00:09.978 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.53613.log: /var/log/ceph/cluster2-client.mirror.62464.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68343.log 2024-04-23T22:00:09.978 INFO:teuthology.orchestra.run.smithi150.stderr: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.62464.log.gz/var/log/ceph/cluster2-client.mirror.50669.log: 2024-04-23T22:00:09.978 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38525.log 2024-04-23T22:00:09.978 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.39615.log 2024-04-23T22:00:09.978 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.38525.log: /var/log/ceph/cluster2-client.mirror.68343.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92665.log 2024-04-23T22:00:09.978 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.39615.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.68343.log.gz 2024-04-23T22:00:09.978 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38323.log 2024-04-23T22:00:09.978 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105446.log 2024-04-23T22:00:09.978 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.38323.log: /var/log/ceph/cluster2-client.mirror.92665.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105265.log 2024-04-23T22:00:09.978 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.92665.log.gz 2024-04-23T22:00:09.978 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91555.log 2024-04-23T22:00:09.979 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105265.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.38525.log.gz 2024-04-23T22:00:09.979 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.42829.log.gz 2024-04-23T22:00:09.981 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.56183.log.gz 2024-04-23T22:00:09.981 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.38323.log.gz 2024-04-23T22:00:09.981 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105446.log: 93.3% -- replaced with /var/log/ceph/cluster2-client.mirror.57857.log.gz 2024-04-23T22:00:09.981 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.105446.log.gz 2024-04-23T22:00:09.981 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57599.log 2024-04-23T22:00:09.982 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.91555.log: 92.0% -- replaced with /var/log/ceph/cluster2-client.mirror.91555.log.gz 2024-04-23T22:00:09.982 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94629.log 2024-04-23T22:00:09.983 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57599.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85160.log 2024-04-23T22:00:09.985 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.94629.log: 91.6% -- replaced with /var/log/ceph/cluster1-client.mirror.94629.log.gz 2024-04-23T22:00:09.988 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.53613.log.gz 2024-04-23T22:00:09.988 INFO:teuthology.orchestra.run.smithi150.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.56488.log.gz 2024-04-23T22:00:09.988 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69980.log 2024-04-23T22:00:09.989 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85160.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.85160.log.gz 2024-04-23T22:00:09.989 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95998.log 2024-04-23T22:00:09.995 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69980.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.50669.log.gz 2024-04-23T22:00:09.995 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.39615.log.gz 2024-04-23T22:00:10.002 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99289.log 2024-04-23T22:00:10.004 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.95998.log: 92.1% -- replaced with /var/log/ceph/cluster1-client.mirror.95998.log.gz 2024-04-23T22:00:10.018 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59147.log 2024-04-23T22:00:10.018 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99289.log: 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.99289.log.gz 2024-04-23T22:00:10.025 INFO:teuthology.orchestra.run.smithi150.stderr: 92.8% -- replaced with /var/log/ceph/cluster1-client.mirror.57456.log.gz 2024-04-23T22:00:10.030 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96761.log 2024-04-23T22:00:10.037 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.59147.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85415.log 2024-04-23T22:00:10.046 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.96761.log: 84.6% -- replaced with /var/log/ceph/cluster2-client.mirror.59147.log.gz 2024-04-23T22:00:10.050 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70995.log 2024-04-23T22:00:10.051 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85415.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.85415.log.gz 2024-04-23T22:00:10.052 INFO:teuthology.orchestra.run.smithi150.stderr: 91.9% -- replaced with /var/log/ceph/cluster2-client.mirror.96761.log.gz 2024-04-23T22:00:10.052 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% -- replaced with /var/log/ceph/cluster1-client.mirror.57619.log.gz 2024-04-23T22:00:10.064 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104536.log 2024-04-23T22:00:10.064 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.70995.log: 87.9% -- replaced with /var/log/ceph/cluster1-client.mirror.70995.log.gz 2024-04-23T22:00:10.075 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77604.log 2024-04-23T22:00:10.076 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104536.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.104536.log.gz 2024-04-23T22:00:10.084 INFO:teuthology.orchestra.run.smithi150.stderr: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.0.37245.log.gz 2024-04-23T22:00:10.086 INFO:teuthology.orchestra.run.smithi150.stderr: 95.4% -- replaced with /var/log/ceph/cluster2-client.mirror.104119.log.gz 2024-04-23T22:00:10.087 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41978.log 2024-04-23T22:00:10.091 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77604.log: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.65148.log.gz 2024-04-23T22:00:10.092 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89947.log 2024-04-23T22:00:10.093 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41978.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58986.log 2024-04-23T22:00:10.097 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89947.log: 92.1% -- replaced with /var/log/ceph/cluster2-client.mirror.89947.log.gz 2024-04-23T22:00:10.097 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89633.log 2024-04-23T22:00:10.097 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58986.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55293.log 2024-04-23T22:00:10.099 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89633.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.89633.log.gz 2024-04-23T22:00:10.099 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.103459.log 2024-04-23T22:00:10.099 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55293.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46242.log 2024-04-23T22:00:10.100 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103459.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.103459.log.gz 2024-04-23T22:00:10.100 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% -- replaced with /var/log/ceph/cluster1-client.mirror.57533.log.gz 2024-04-23T22:00:10.105 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.55293.log.gz 2024-04-23T22:00:10.113 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-osd.1.log 2024-04-23T22:00:10.113 INFO:teuthology.orchestra.run.smithi150.stderr: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.58986.log.gz 2024-04-23T22:00:10.128 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46242.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102023.log 2024-04-23T22:00:10.136 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-osd.1.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50850.log 2024-04-23T22:00:10.136 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102023.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.0.35446.log 2024-04-23T22:00:10.136 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102023.log.gz 2024-04-23T22:00:10.139 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50850.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101866.log 2024-04-23T22:00:10.139 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.0.35446.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55680.log 2024-04-23T22:00:10.139 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101866.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101866.log.gz 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.69980.log.gz 2024-04-23T22:00:10.141 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88234.log 2024-04-23T22:00:10.141 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55680.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73450.log 2024-04-23T22:00:10.142 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88234.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.88234.log.gz 2024-04-23T22:00:10.142 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:10.142 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34479.log 2024-04-23T22:00:10.144 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73450.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.73450.log.gz 2024-04-23T22:00:10.144 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99473.log 2024-04-23T22:00:10.144 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34479.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34479.log.gz 2024-04-23T22:00:10.147 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57123.log 2024-04-23T22:00:10.147 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99473.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.99473.log.gz 2024-04-23T22:00:10.147 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103212.log 2024-04-23T22:00:10.147 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.2.78376.log 2024-04-23T22:00:10.147 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57123.log: 70.9% -- replaced with /var/log/ceph/cluster1-client.mirror.0.35446.log.gz 2024-04-23T22:00:10.147 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45067.log 2024-04-23T22:00:10.148 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.2.78376.log: /var/log/ceph/cluster2-client.mirror.103212.log: 54.4% -- replaced with /var/log/ceph/cluster2-client.mirror.103212.log.gz 2024-04-23T22:00:10.158 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105941.log 2024-04-23T22:00:10.158 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.45067.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105996.log 2024-04-23T22:00:10.163 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.105941.log: 88.8% -- replaced with /var/log/ceph/cluster1-client.mirror.57123.log.gz 2024-04-23T22:00:10.165 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.45067.log.gz 2024-04-23T22:00:10.165 INFO:teuthology.orchestra.run.smithi150.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.105941.log.gz 2024-04-23T22:00:10.175 INFO:teuthology.orchestra.run.smithi150.stderr: 93.6% -- replaced with /var/log/ceph/cluster1-client.mirror.2.78376.log.gz 2024-04-23T22:00:10.176 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83308.log 2024-04-23T22:00:10.178 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71715.log 2024-04-23T22:00:10.178 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.83308.log: /var/log/ceph/cluster1-client.mirror.105996.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.105996.log.gz 2024-04-23T22:00:10.178 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40717.log 2024-04-23T22:00:10.181 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71715.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.71715.log.gz 2024-04-23T22:00:10.181 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40911.log 2024-04-23T22:00:10.181 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40717.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71259.log 2024-04-23T22:00:10.181 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40911.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106400.log 2024-04-23T22:00:10.182 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71259.log: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.61414.log.gz 2024-04-23T22:00:10.182 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.71259.log.gz 2024-04-23T22:00:10.187 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.40717.log.gz 2024-04-23T22:00:10.188 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96264.log 2024-04-23T22:00:10.189 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106400.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.40911.log.gz 2024-04-23T22:00:10.189 INFO:teuthology.orchestra.run.smithi150.stderr: 83.4% -- replaced with /var/log/ceph/cluster2-client.mirror.106400.log.gz 2024-04-23T22:00:10.189 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89363.log 2024-04-23T22:00:10.190 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.96264.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.96264.log.gz 2024-04-23T22:00:10.191 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77030.log 2024-04-23T22:00:10.191 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89363.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.89363.log.gz 2024-04-23T22:00:10.192 INFO:teuthology.orchestra.run.smithi150.stderr: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.83308.log.gzgzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68639.log 2024-04-23T22:00:10.192 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:10.192 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77030.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.103348.log 2024-04-23T22:00:10.193 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68639.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.68639.log.gz 2024-04-23T22:00:10.193 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70048.log 2024-04-23T22:00:10.194 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103348.log: 76.8% -- replaced with /var/log/ceph/cluster1-client.mirror.103348.log.gz 2024-04-23T22:00:10.194 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45358.log 2024-04-23T22:00:10.198 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.70048.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.70048.log.gz 2024-04-23T22:00:10.198 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42234.log 2024-04-23T22:00:10.198 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.39257.log 2024-04-23T22:00:10.198 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.42234.log: /var/log/ceph/cluster2-client.mirror.45358.log: gzip -5 91.9% --verbose -- -- replaced with /var/log/ceph/cluster2-client.mirror.52568.log.gz /var/log/ceph/cluster2-client.mirror.100825.log 2024-04-23T22:00:10.198 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:10.198 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92764.log 2024-04-23T22:00:10.199 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100825.log: /var/log/ceph/cluster2-client.mirror.39257.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58006.log 2024-04-23T22:00:10.199 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.100825.log.gz 2024-04-23T22:00:10.208 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.92764.log: 97.1% -- replaced with /var/log/ceph/cluster2-client.mirror.92764.log.gz 2024-04-23T22:00:10.209 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78843.log 2024-04-23T22:00:10.209 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.58006.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61964.log 2024-04-23T22:00:10.209 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.60896.log.gz 2024-04-23T22:00:10.213 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.78843.log: 94.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78843.log.gz 2024-04-23T22:00:10.213 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102473.log 2024-04-23T22:00:10.213 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/cluster2-client.mirror.61964.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65763.log 2024-04-23T22:00:10.213 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102473.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43702.log 2024-04-23T22:00:10.214 INFO:teuthology.orchestra.run.smithi150.stderr: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.102473.log.gz 2024-04-23T22:00:10.214 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.65763.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.65763.log.gz 2024-04-23T22:00:10.221 INFO:teuthology.orchestra.run.smithi150.stderr: 95.5% -- replaced with /var/log/ceph/cluster2-client.mirror.77604.log.gz 2024-04-23T22:00:10.227 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49944.log 2024-04-23T22:00:10.227 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.45358.log.gz 2024-04-23T22:00:10.227 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.43702.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42467.log 2024-04-23T22:00:10.227 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.49944.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44287.log 2024-04-23T22:00:10.228 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.42467.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97330.log 2024-04-23T22:00:10.228 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/cluster2-client.mirror.44287.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79578.log 2024-04-23T22:00:10.229 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97330.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.97330.log.gz 2024-04-23T22:00:10.267 INFO:teuthology.orchestra.run.smithi148.stderr: 91.1% -- replaced with /var/log/ceph/cluster1-mgr.x.log.gz 2024-04-23T22:00:10.291 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82973.log 2024-04-23T22:00:10.291 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.79578.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43931.log 2024-04-23T22:00:10.291 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.82973.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.79578.log.gz 2024-04-23T22:00:10.291 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26282.log 2024-04-23T22:00:10.291 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.82973.log.gz 2024-04-23T22:00:10.291 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.43931.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.103036.log 2024-04-23T22:00:10.291 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph.tmp-client.admin.26282.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26282.log.gz 2024-04-23T22:00:10.291 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.37611.log 2024-04-23T22:00:10.291 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103036.log: gzip 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.103036.log.gz 2024-04-23T22:00:10.291 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96399.log 2024-04-23T22:00:10.291 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.37611.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106718.log 2024-04-23T22:00:10.291 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.96399.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59929.log 2024-04-23T22:00:10.292 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.96399.log.gz 2024-04-23T22:00:10.292 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.106718.log: gzip -5 --verbose -- 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.106718.log.gz 2024-04-23T22:00:10.292 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster1-client.mirror.57834.log 2024-04-23T22:00:10.292 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.59929.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44206.log 2024-04-23T22:00:10.292 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57834.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.3.35645.log 2024-04-23T22:00:10.292 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44206.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70475.log 2024-04-23T22:00:10.292 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97981.log 2024-04-23T22:00:10.292 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.3.35645.log: /var/log/ceph/cluster2-client.mirror.70475.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71808.log 2024-04-23T22:00:10.292 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97981.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80126.log 2024-04-23T22:00:10.292 INFO:teuthology.orchestra.run.smithi150.stderr: 91.7%/var/log/ceph/cluster2-client.mirror.71808.log: -- replaced with /var/log/ceph/cluster2-client.mirror.70475.log.gz 2024-04-23T22:00:10.292 INFO:teuthology.orchestra.run.smithi150.stderr: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.97981.log.gz 2024-04-23T22:00:10.292 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.71808.log.gz 2024-04-23T22:00:10.292 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93356.log 2024-04-23T22:00:10.293 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75069.log 2024-04-23T22:00:10.293 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.80126.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40329.log 2024-04-23T22:00:10.293 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9%/var/log/ceph/cluster2-client.mirror.75069.log: -- replaced with /var/log/ceph/cluster2-client.mirror.80126.log.gz 2024-04-23T22:00:10.293 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.75069.log.gz/var/log/ceph/cluster2-client.mirror.93356.log: 2024-04-23T22:00:10.293 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79909.log 2024-04-23T22:00:10.293 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.93356.log.gz 2024-04-23T22:00:10.293 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40329.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93161.log 2024-04-23T22:00:10.293 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.79909.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98161.log 2024-04-23T22:00:10.293 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.79909.log.gz 2024-04-23T22:00:10.293 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.93161.log: gzip 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.93161.log.gz 2024-04-23T22:00:10.293 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106933.log 2024-04-23T22:00:10.293 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58598.log 2024-04-23T22:00:10.293 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106933.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106933.log.gz 2024-04-23T22:00:10.293 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98161.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.98161.log.gz 2024-04-23T22:00:10.294 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72640.log 2024-04-23T22:00:10.294 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/cluster1-client.mirror.58598.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87765.log 2024-04-23T22:00:10.294 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.72640.log: gzip -5 --verbose -- 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.72640.log.gz 2024-04-23T22:00:10.294 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster2-client.mirror.92272.log 2024-04-23T22:00:10.294 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87765.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71162.log 2024-04-23T22:00:10.294 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.87765.log.gz 2024-04-23T22:00:10.294 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.92272.log: gzip -5 --verbose 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.92272.log.gz 2024-04-23T22:00:10.294 INFO:teuthology.orchestra.run.smithi150.stderr: -- /var/log/ceph/cluster2-client.mirror.67969.log 2024-04-23T22:00:10.294 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71162.log: 87.1% -- replaced with /var/log/ceph/cluster2-client.mirror.71162.log.gz/var/log/ceph/cluster2-client.mirror.67969.log: 2024-04-23T22:00:10.294 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102203.log 2024-04-23T22:00:10.294 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58565.log 2024-04-23T22:00:10.294 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102203.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.102203.log.gz 2024-04-23T22:00:10.295 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.104602.log 2024-04-23T22:00:10.295 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.43702.log.gz 2024-04-23T22:00:10.295 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.43931.log.gz 2024-04-23T22:00:10.295 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58565.log: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.77030.log.gz 2024-04-23T22:00:10.295 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85280.log 2024-04-23T22:00:10.295 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73206.log 2024-04-23T22:00:10.295 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.104602.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72374.log 2024-04-23T22:00:10.295 INFO:teuthology.orchestra.run.smithi150.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.104602.log.gz 2024-04-23T22:00:10.295 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73206.log: /var/log/ceph/cluster2-client.mirror.85280.log: gzip 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.73206.log.gz 2024-04-23T22:00:10.295 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59994.log 2024-04-23T22:00:10.295 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9%/var/log/ceph/cluster1-client.mirror.72374.log: -- replaced with /var/log/ceph/cluster2-client.mirror.85280.log.gz 2024-04-23T22:00:10.295 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101596.log 2024-04-23T22:00:10.295 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5/var/log/ceph/cluster2-client.mirror.59994.log: --verbose -- /var/log/ceph/cluster2-client.mirror.2.35627.log 2024-04-23T22:00:10.295 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101596.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81823.log 2024-04-23T22:00:10.295 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101596.log.gz 2024-04-23T22:00:10.296 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.2.35627.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101731.log 2024-04-23T22:00:10.296 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81823.log: 87.8% 91.7% -- replaced with /var/log/ceph/cluster1-client.mirror.72374.log.gz 2024-04-23T22:00:10.296 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.81823.log.gz 2024-04-23T22:00:10.296 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76963.log 2024-04-23T22:00:10.296 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101731.log: gzip -5 --verbose -- 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101731.log.gz 2024-04-23T22:00:10.296 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster2-client.mirror.105780.log 2024-04-23T22:00:10.296 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/cluster1-client.mirror.76963.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85831.log 2024-04-23T22:00:10.296 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76963.log.gz 2024-04-23T22:00:10.296 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105780.log: 83.3%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62378.log 2024-04-23T22:00:10.296 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.105780.log.gz 2024-04-23T22:00:10.296 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85831.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.58598.log.gz 2024-04-23T22:00:10.296 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.85831.log.gz 2024-04-23T22:00:10.296 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72810.log 2024-04-23T22:00:10.297 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.62378.log: gzip -5 --verbose -- 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.62378.log.gz 2024-04-23T22:00:10.297 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster1-client.mirror.105092.log 2024-04-23T22:00:10.297 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/cluster2-client.mirror.72810.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47664.log 2024-04-23T22:00:10.297 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.72810.log.gz 2024-04-23T22:00:10.297 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.105092.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.105092.log.gz 2024-04-23T22:00:10.297 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38081.log 2024-04-23T22:00:10.297 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/cluster2-client.mirror.47664.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38692.log 2024-04-23T22:00:10.297 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70450.log 2024-04-23T22:00:10.297 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.38692.log: /var/log/ceph/cluster2-client.mirror.38081.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101551.log 2024-04-23T22:00:10.297 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.70450.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53192.log 2024-04-23T22:00:10.297 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70450.log.gz 2024-04-23T22:00:10.297 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101551.log: 86.8% -- replaced with /var/log/ceph/cluster1-client.mirror.101551.log.gz 2024-04-23T22:00:10.297 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57192.log 2024-04-23T22:00:10.297 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.53192.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100988.log 2024-04-23T22:00:10.298 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57192.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46597.log 2024-04-23T22:00:10.298 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.41978.log.gz 2024-04-23T22:00:10.298 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.100988.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62617.log 2024-04-23T22:00:10.298 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.42234.log.gz 2024-04-23T22:00:10.298 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.100988.log.gz 2024-04-23T22:00:10.298 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46597.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.42467.log.gz 2024-04-23T22:00:10.298 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97187.log 2024-04-23T22:00:10.298 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.62617.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90642.log 2024-04-23T22:00:10.298 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97187.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.97187.log.gz 2024-04-23T22:00:10.298 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85947.log 2024-04-23T22:00:10.298 INFO:teuthology.orchestra.run.smithi150.stderr: 93.7%/var/log/ceph/cluster1-client.mirror.90642.log: -- replaced with /var/log/ceph/cluster2-client.mirror.35508.log.gz 2024-04-23T22:00:10.298 INFO:teuthology.orchestra.run.smithi150.stderr:gzip 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.90642.log.gz 2024-04-23T22:00:10.298 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.36431.log 2024-04-23T22:00:10.298 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52246.log 2024-04-23T22:00:10.299 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85947.log: /var/log/ceph/cluster2-client.mirror.36431.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46729.log 2024-04-23T22:00:10.299 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52246.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.85947.log.gz 2024-04-23T22:00:10.299 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98010.log 2024-04-23T22:00:10.299 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92630.log 2024-04-23T22:00:10.299 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98010.log: /var/log/ceph/cluster2-client.mirror.46729.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40585.log 2024-04-23T22:00:10.299 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.98010.log.gz 2024-04-23T22:00:10.299 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.92630.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51657.log 2024-04-23T22:00:10.299 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40585.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.92630.log.gz 2024-04-23T22:00:10.299 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.36173.log 2024-04-23T22:00:10.299 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105157.log 2024-04-23T22:00:10.299 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.36173.log: /var/log/ceph/cluster2-client.mirror.51657.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99522.log 2024-04-23T22:00:10.299 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.31343.log 2024-04-23T22:00:10.299 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99522.log: /var/log/ceph/cluster2-client.mirror.105157.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.105157.log.gz 2024-04-23T22:00:10.300 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.99522.log.gz 2024-04-23T22:00:10.307 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.49944.log.gz 2024-04-23T22:00:10.307 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.31343.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.31343.log.gzgzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58157.log 2024-04-23T22:00:10.307 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:10.307 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.37611.log.gz 2024-04-23T22:00:10.307 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.40329.log.gz 2024-04-23T22:00:10.307 INFO:teuthology.orchestra.run.smithi150.stderr: 84.6%/var/log/ceph/cluster2-client.mirror.58157.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.40585.log.gz 2024-04-23T22:00:10.307 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65268.log 2024-04-23T22:00:10.308 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.59994.log.gz 2024-04-23T22:00:10.308 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73958.log 2024-04-23T22:00:10.308 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.65268.log: 79.9% -- replaced with /var/log/ceph/cluster2-client.admin.36173.log.gz 2024-04-23T22:00:10.308 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.65268.log.gz 2024-04-23T22:00:10.308 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100901.log 2024-04-23T22:00:10.308 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73958.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.73958.log.gz 2024-04-23T22:00:10.308 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51506.log 2024-04-23T22:00:10.308 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.100901.log: 1.3% -- replaced with /var/log/ceph/cluster1-client.mirror.100901.log.gz 2024-04-23T22:00:10.308 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82795.log 2024-04-23T22:00:10.308 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.51506.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56094.log 2024-04-23T22:00:10.308 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.82795.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.82795.log.gz 2024-04-23T22:00:10.308 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41202.log 2024-04-23T22:00:10.310 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56094.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89180.log 2024-04-23T22:00:10.310 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41202.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72081.log 2024-04-23T22:00:10.310 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.89180.log: 92.0% -- replaced with /var/log/ceph/cluster1-client.mirror.89180.log.gz 2024-04-23T22:00:10.312 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.38081.log.gz 2024-04-23T22:00:10.312 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.38692.log.gz 2024-04-23T22:00:10.316 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43120.log 2024-04-23T22:00:10.316 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.72081.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.72081.log.gz 2024-04-23T22:00:10.316 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68510.log 2024-04-23T22:00:10.316 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.43120.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46442.log 2024-04-23T22:00:10.316 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68510.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.68510.log.gz 2024-04-23T22:00:10.320 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster2-client.mirror.46442.log: -- /var/log/ceph/cluster2-client.mirror.64766.log 2024-04-23T22:00:10.320 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71062.log 2024-04-23T22:00:10.321 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.64766.log: 1.3% -- replaced with /var/log/ceph/cluster2-client.mirror.64766.log.gz 2024-04-23T22:00:10.321 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48572.log 2024-04-23T22:00:10.332 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71062.log: 88.2% 54.2% -- replaced with /var/log/ceph/cluster2-client.mirror.36431.log.gz 2024-04-23T22:00:10.332 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.47664.log.gz 2024-04-23T22:00:10.332 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71569.log 2024-04-23T22:00:10.338 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48572.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73376.log 2024-04-23T22:00:10.343 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.71569.log: 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.71569.log.gz 2024-04-23T22:00:10.345 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.92697.log 2024-04-23T22:00:10.355 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.73376.log: 92.0% 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.44206.log.gz 2024-04-23T22:00:10.355 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.73376.log.gz 2024-04-23T22:00:10.356 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.44287.log.gz 2024-04-23T22:00:10.359 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.41202.log.gz 2024-04-23T22:00:10.360 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58708.log 2024-04-23T22:00:10.361 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.92697.log: 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.58157.log.gz 2024-04-23T22:00:10.369 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.39809.log 2024-04-23T22:00:10.373 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58708.log: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.105265.log.gz 2024-04-23T22:00:10.373 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56094.log.gz 2024-04-23T22:00:10.374 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.43120.log.gz 88.0% 2024-04-23T22:00:10.374 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.92697.log.gz 2024-04-23T22:00:10.376 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51108.log 2024-04-23T22:00:10.381 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.39809.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.48572.log.gz 2024-04-23T22:00:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.45425.log 2024-04-23T22:00:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.51108.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69485.log 2024-04-23T22:00:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.45425.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102271.log 2024-04-23T22:00:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69485.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.52246.log.gz 2024-04-23T22:00:10.388 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.69485.log.gz 2024-04-23T22:00:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.107320.log 2024-04-23T22:00:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102271.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102271.log.gz 2024-04-23T22:00:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73663.log 2024-04-23T22:00:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.107320.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.107320.log.gz 2024-04-23T22:00:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71502.log 2024-04-23T22:00:10.388 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73663.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.73663.log.gz 2024-04-23T22:00:10.389 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84448.log 2024-04-23T22:00:10.389 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71502.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.71502.log.gz 2024-04-23T22:00:10.390 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.53192.log.gz 2024-04-23T22:00:10.390 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87632.log 2024-04-23T22:00:10.390 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.84448.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.84448.log.gz 2024-04-23T22:00:10.397 INFO:teuthology.orchestra.run.smithi150.stderr: 93.8% -- replaced with /var/log/ceph/cluster2-client.mirror.2.57401.log.gz 2024-04-23T22:00:10.397 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88968.log 2024-04-23T22:00:10.399 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58267.log 2024-04-23T22:00:10.399 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87632.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.87632.log.gz 2024-04-23T22:00:10.399 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100654.log 2024-04-23T22:00:10.400 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58267.log: /var/log/ceph/cluster2-client.mirror.88968.log: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.52381.log.gz 2024-04-23T22:00:10.400 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65125.log 2024-04-23T22:00:10.404 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100654.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.100654.log.gz 2024-04-23T22:00:10.404 INFO:teuthology.orchestra.run.smithi150.stderr: 91.0% -- replaced with /var/log/ceph/cluster2-client.mirror.58565.log.gzgzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52313.log 2024-04-23T22:00:10.404 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:10.405 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.65125.log: 1.3% -- replaced with /var/log/ceph/cluster1-client.mirror.65125.log.gz 2024-04-23T22:00:10.405 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57814.log 2024-04-23T22:00:10.406 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.52313.log: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.51108.log.gz 2024-04-23T22:00:10.407 INFO:teuthology.orchestra.run.smithi150.stderr: 89.4% -- replaced with /var/log/ceph/cluster2-client.mirror.58267.log.gz 2024-04-23T22:00:10.410 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster1-client.mirror.45425.log.gz 2024-04-23T22:00:10.413 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80896.log 2024-04-23T22:00:10.414 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57814.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69584.log 2024-04-23T22:00:10.415 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.80896.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.80896.log.gz 2024-04-23T22:00:10.415 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.1.35591.log 2024-04-23T22:00:10.424 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69584.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60352.log 2024-04-23T22:00:10.433 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.1.35591.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74375.log 2024-04-23T22:00:10.440 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60352.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.39809.log.gz 2024-04-23T22:00:10.446 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% -- replaced with /var/log/ceph/cluster1-client.mirror.57834.log.gz 2024-04-23T22:00:10.446 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76047.log 2024-04-23T22:00:10.447 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74375.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.74375.log.gz 2024-04-23T22:00:10.460 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87797.log 2024-04-23T22:00:10.461 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76047.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.76047.log.gz 2024-04-23T22:00:10.477 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70083.log 2024-04-23T22:00:10.478 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.87797.log: 88.2% -- replaced with /var/log/ceph/cluster1-client.mirror.87797.log.gz 2024-04-23T22:00:10.489 INFO:teuthology.orchestra.run.smithi150.stderr: 96.8% -- replaced with /var/log/ceph/cluster2-client.mirror.55680.log.gz 2024-04-23T22:00:10.489 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96829.log 2024-04-23T22:00:10.489 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.70083.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.70083.log.gz 2024-04-23T22:00:10.498 INFO:teuthology.orchestra.run.smithi150.stderr: 88.5% -- replaced with /var/log/ceph/cluster1-client.mirror.52313.log.gz 2024-04-23T22:00:10.500 INFO:teuthology.orchestra.run.smithi150.stderr: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.60352.log.gz 2024-04-23T22:00:10.505 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.62617.log.gz 2024-04-23T22:00:10.513 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59821.log 2024-04-23T22:00:10.514 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.96829.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.96829.log.gz 2024-04-23T22:00:10.523 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57599.log.gz 2024-04-23T22:00:10.526 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50765.log 2024-04-23T22:00:10.527 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.59821.log: 95.5% -- replaced with /var/log/ceph/cluster2-client.mirror.88968.log.gz 2024-04-23T22:00:10.528 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40814.log 2024-04-23T22:00:10.529 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50765.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.37677.log 2024-04-23T22:00:10.529 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40814.log: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.69584.log.gz 2024-04-23T22:00:10.530 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72943.log 2024-04-23T22:00:10.534 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose --/var/log/ceph/cluster2-client.mirror.37677.log: /var/log/ceph/cluster2-client.mirror.79794.log 2024-04-23T22:00:10.534 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.36147.log 2024-04-23T22:00:10.534 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.79794.log: /var/log/ceph/cluster2-client.mirror.72943.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.79794.log.gz 2024-04-23T22:00:10.534 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.72943.log.gz 2024-04-23T22:00:10.534 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.36346.log 2024-04-23T22:00:10.534 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.36147.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.36147.log.gz 2024-04-23T22:00:10.535 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79934.log 2024-04-23T22:00:10.535 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9%/var/log/ceph/cluster2-client.mirror.36346.log: -- replaced with /var/log/ceph/cluster2-client.mirror.37677.log.gz 2024-04-23T22:00:10.535 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97825.log 2024-04-23T22:00:10.535 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51305.log 2024-04-23T22:00:10.535 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.79934.log: /var/log/ceph/cluster2-client.mirror.97825.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.97825.log.gz 2024-04-23T22:00:10.542 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48378.log 2024-04-23T22:00:10.542 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.51305.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.2.107090.log 2024-04-23T22:00:10.542 INFO:teuthology.orchestra.run.smithi150.stderr: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.59821.log.gz 2024-04-23T22:00:10.542 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48378.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99408.log 2024-04-23T22:00:10.542 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.2.107090.log: 93.2% 80.7% -- replaced with /var/log/ceph/cluster2-client.mirror.2.107090.log.gz 2024-04-23T22:00:10.542 INFO:teuthology.orchestra.run.smithi150.stderr: 92.0% -- replaced with /var/log/ceph/cluster2-client.mirror.79934.log.gz 2024-04-23T22:00:10.542 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.57814.log.gz 2024-04-23T22:00:10.542 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41784.log 2024-04-23T22:00:10.543 INFO:teuthology.orchestra.run.smithi150.stderr: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.58708.log.gz 2024-04-23T22:00:10.543 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99408.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.99408.log.gz 2024-04-23T22:00:10.543 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88699.log 2024-04-23T22:00:10.543 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41784.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79346.log 2024-04-23T22:00:10.543 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88699.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81354.log 2024-04-23T22:00:10.543 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.88699.log.gz 2024-04-23T22:00:10.543 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.79346.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64503.log 2024-04-23T22:00:10.544 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81354.log: 91.6% -- replaced with /var/log/ceph/cluster2-client.mirror.81354.log.gz 2024-04-23T22:00:10.544 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53087.log 2024-04-23T22:00:10.546 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.64503.log: 88.0% 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.41784.log.gz 2024-04-23T22:00:10.546 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.51305.log.gz 2024-04-23T22:00:10.551 INFO:teuthology.orchestra.run.smithi150.stderr: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.64503.log.gz 2024-04-23T22:00:10.551 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77470.log 2024-04-23T22:00:10.553 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.53087.log: /var/log/ceph/cluster2-client.mirror.77470.log: 87.8%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98776.log 2024-04-23T22:00:10.554 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.77470.log.gz 2024-04-23T22:00:10.554 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93824.log 2024-04-23T22:00:10.555 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98776.log: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.98776.log.gz 2024-04-23T22:00:10.555 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94010.log 2024-04-23T22:00:10.555 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.93824.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.93824.log.gz 2024-04-23T22:00:10.559 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81571.log 2024-04-23T22:00:10.559 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.94010.log: 87.9% -- replaced with /var/log/ceph/cluster1-client.mirror.94010.log.gz 2024-04-23T22:00:10.560 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.107052.log 2024-04-23T22:00:10.560 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.81571.log: 87.9% -- replaced with /var/log/ceph/cluster1-client.mirror.81571.log.gz 2024-04-23T22:00:10.560 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76284.log 2024-04-23T22:00:10.560 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.107052.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.107052.log.gz 2024-04-23T22:00:10.560 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106983.log 2024-04-23T22:00:10.560 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76284.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.76284.log.gz 2024-04-23T22:00:10.561 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106983.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106983.log.gz 2024-04-23T22:00:10.561 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100634.log 2024-04-23T22:00:10.561 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.48378.log.gz 2024-04-23T22:00:10.563 INFO:teuthology.orchestra.run.smithi150.stderr: 5.8% -- replaced with /var/log/ceph/cluster2-client.mirror.36346.log.gz 2024-04-23T22:00:10.564 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95393.log 2024-04-23T22:00:10.570 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100634.log: 92.8% -- replaced with /var/log/ceph/cluster1-client.mirror.45513.log.gz 2024-04-23T22:00:10.570 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.100634.log.gz 2024-04-23T22:00:10.571 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86509.log 2024-04-23T22:00:10.572 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95393.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.95393.log.gz 2024-04-23T22:00:10.572 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98320.log 2024-04-23T22:00:10.572 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86509.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.86509.log.gz 2024-04-23T22:00:10.573 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97868.log 2024-04-23T22:00:10.574 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98320.log: 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.98320.log.gz 2024-04-23T22:00:10.574 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50954.log 2024-04-23T22:00:10.575 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97868.log: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.97868.log.gz 2024-04-23T22:00:10.576 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88153.log 2024-04-23T22:00:10.576 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50954.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90667.log 2024-04-23T22:00:10.577 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88153.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.88153.log.gz 2024-04-23T22:00:10.577 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57791.log 2024-04-23T22:00:10.581 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90667.log: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.46597.log.gz 2024-04-23T22:00:10.581 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.46729.log.gz 2024-04-23T22:00:10.584 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.40814.log.gz 2024-04-23T22:00:10.584 INFO:teuthology.orchestra.run.smithi150.stderr: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.90667.log.gz 2024-04-23T22:00:10.584 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47567.log 2024-04-23T22:00:10.584 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57791.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71680.log 2024-04-23T22:00:10.585 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.47567.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60375.log 2024-04-23T22:00:10.585 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71680.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.71680.log.gz 2024-04-23T22:00:10.586 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102563.log 2024-04-23T22:00:10.589 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.47567.log.gz 2024-04-23T22:00:10.591 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60375.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106317.log 2024-04-23T22:00:10.591 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102563.log: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.102563.log.gz 2024-04-23T22:00:10.594 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101776.log 2024-04-23T22:00:10.594 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.106317.log: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.106317.log.gz 2024-04-23T22:00:10.594 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48281.log 2024-04-23T22:00:10.594 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101776.log: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.67969.log.gz 2024-04-23T22:00:10.594 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101776.log.gz 2024-04-23T22:00:10.594 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57900.log 2024-04-23T22:00:10.597 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48281.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.107780.log 2024-04-23T22:00:10.597 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57900.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62594.log 2024-04-23T22:00:10.598 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.107780.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.107780.log.gz 2024-04-23T22:00:10.598 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103586.log 2024-04-23T22:00:10.598 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.62594.log: 84.5% -- replaced with /var/log/ceph/cluster2-client.mirror.62594.log.gz 2024-04-23T22:00:10.598 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.2.56767.log 2024-04-23T22:00:10.602 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.103586.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.103586.log.gz 2024-04-23T22:00:10.602 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80989.log 2024-04-23T22:00:10.602 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.2.56767.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102428.log 2024-04-23T22:00:10.602 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.80989.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56613.log 2024-04-23T22:00:10.602 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.80989.log.gz 2024-04-23T22:00:10.602 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102428.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92183.log 2024-04-23T22:00:10.602 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56613.log: 86.8% -- replaced with /var/log/ceph/cluster1-client.mirror.102428.log.gz 2024-04-23T22:00:10.605 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66216.log 2024-04-23T22:00:10.605 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.92183.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96349.log 2024-04-23T22:00:10.605 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.92183.log.gz 2024-04-23T22:00:10.605 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66216.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.66216.log.gz 2024-04-23T22:00:10.605 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.56613.log.gz 2024-04-23T22:00:10.608 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100725.log 2024-04-23T22:00:10.608 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.96349.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41458.log 2024-04-23T22:00:10.608 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.96349.log.gz 2024-04-23T22:00:10.609 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100725.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.48281.log.gz 2024-04-23T22:00:10.610 INFO:teuthology.orchestra.run.smithi150.stderr: 90.4% -- replaced with /var/log/ceph/cluster2-client.mirror.100725.log.gz 2024-04-23T22:00:10.613 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84890.log 2024-04-23T22:00:10.613 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72993.log 2024-04-23T22:00:10.614 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.84890.log: 91.8% -- replaced with /var/log/ceph/cluster1-client.mirror.84890.log.gz 2024-04-23T22:00:10.617 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41458.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102136.log 2024-04-23T22:00:10.617 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.72993.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60299.log 2024-04-23T22:00:10.617 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.72993.log.gz 2024-04-23T22:00:10.617 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102136.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.102136.log.gz 2024-04-23T22:00:10.632 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101528.log 2024-04-23T22:00:10.632 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60299.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43508.log 2024-04-23T22:00:10.632 INFO:teuthology.orchestra.run.smithi150.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.58006.log.gz 2024-04-23T22:00:10.632 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101528.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.78308.log 2024-04-23T22:00:10.632 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.43508.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.101528.log.gz 2024-04-23T22:00:10.632 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.35361.log 2024-04-23T22:00:10.632 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.78308.log: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.79346.log.gz 2024-04-23T22:00:10.632 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95978.log 2024-04-23T22:00:10.632 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.78308.log.gz 2024-04-23T22:00:10.633 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.35361.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.35361.log.gz 2024-04-23T22:00:10.633 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.103013.log 2024-04-23T22:00:10.633 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95978.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74123.log 2024-04-23T22:00:10.633 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106081.log 2024-04-23T22:00:10.633 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.74123.log: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.74123.log.gz 2024-04-23T22:00:10.633 INFO:teuthology.orchestra.run.smithi150.stderr: 92.3% -- replaced with /var/log/ceph/cluster2-client.mirror.95978.log.gz 2024-04-23T22:00:10.633 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103013.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95833.log 2024-04-23T22:00:10.633 INFO:teuthology.orchestra.run.smithi150.stderr: 86.9% -- replaced with /var/log/ceph/cluster1-client.mirror.103013.log.gz 2024-04-23T22:00:10.633 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106081.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99924.log 2024-04-23T22:00:10.633 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85195.log 2024-04-23T22:00:10.633 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99924.log: /var/log/ceph/cluster2-client.mirror.95833.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.99924.log.gz 2024-04-23T22:00:10.633 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105115.log 2024-04-23T22:00:10.633 INFO:teuthology.orchestra.run.smithi150.stderr: 91.5%/var/log/ceph/cluster2-client.mirror.85195.log: -- replaced with /var/log/ceph/cluster2-client.mirror.106081.log.gz 2024-04-23T22:00:10.633 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34227.log 2024-04-23T22:00:10.634 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.105115.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.105115.log.gz 2024-04-23T22:00:10.634 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51740.log 2024-04-23T22:00:10.634 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34227.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34227.log.gz 2024-04-23T22:00:10.634 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100470.log 2024-04-23T22:00:10.634 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.85195.log.gz 2024-04-23T22:00:10.634 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.51740.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61522.log 2024-04-23T22:00:10.634 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100470.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44252.log 2024-04-23T22:00:10.634 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.61522.log: 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.100470.log.gz 2024-04-23T22:00:10.636 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84165.log 2024-04-23T22:00:10.636 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44252.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98362.log 2024-04-23T22:00:10.636 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.84165.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.84165.log.gz 2024-04-23T22:00:10.637 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95791.log 2024-04-23T22:00:10.640 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98362.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.98362.log.gz 2024-04-23T22:00:10.649 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105487.log 2024-04-23T22:00:10.649 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.41458.log.gz 2024-04-23T22:00:10.649 INFO:teuthology.orchestra.run.smithi150.stderr: 92.8% -- replaced with /var/log/ceph/cluster2-client.mirror.57192.log.gz 2024-04-23T22:00:10.649 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95791.log: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.60299.log.gz 2024-04-23T22:00:10.650 INFO:teuthology.orchestra.run.smithi150.stderr: 92.1% -- replaced with /var/log/ceph/cluster2-client.mirror.95791.log.gz 2024-04-23T22:00:10.650 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77008.log 2024-04-23T22:00:10.650 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105487.log: 80.9% -- replaced with /var/log/ceph/cluster2-client.mirror.105487.log.gz 2024-04-23T22:00:10.650 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98889.log 2024-04-23T22:00:10.650 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.77008.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.43508.log.gz 2024-04-23T22:00:10.651 INFO:teuthology.orchestra.run.smithi150.stderr: 91.7% -- replaced with /var/log/ceph/cluster1-client.mirror.77008.log.gz 2024-04-23T22:00:10.651 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70153.log 2024-04-23T22:00:10.654 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98889.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.98889.log.gz 2024-04-23T22:00:10.654 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80683.log 2024-04-23T22:00:10.655 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.70153.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.44252.log.gz 2024-04-23T22:00:10.655 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.70153.log.gz 2024-04-23T22:00:10.655 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49022.log 2024-04-23T22:00:10.656 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.80683.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.80683.log.gz 2024-04-23T22:00:10.657 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46104.log 2024-04-23T22:00:10.657 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.49022.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60633.log 2024-04-23T22:00:10.661 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46104.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68825.log 2024-04-23T22:00:10.662 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60633.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57771.log 2024-04-23T22:00:10.672 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68825.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81143.log 2024-04-23T22:00:10.676 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57771.log: 96.5% -- replaced with /var/log/ceph/cluster2-client.mirror.50850.log.gz 2024-04-23T22:00:10.678 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99190.log 2024-04-23T22:00:10.682 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81143.log: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.49022.log.gz 2024-04-23T22:00:10.683 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.81143.log.gz 2024-04-23T22:00:10.683 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.50765.log.gz 2024-04-23T22:00:10.683 INFO:teuthology.orchestra.run.smithi150.stderr: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.46104.log.gz 2024-04-23T22:00:10.695 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83955.log 2024-04-23T22:00:10.695 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% -- replaced with /var/log/ceph/cluster1-client.mirror.51506.log.gz 2024-04-23T22:00:10.696 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99190.log: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.99190.log.gz 2024-04-23T22:00:10.698 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.83955.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105554.log 2024-04-23T22:00:10.698 INFO:teuthology.orchestra.run.smithi150.stderr: 91.8% -- replaced with /var/log/ceph/cluster2-client.mirror.83955.log.gz 2024-04-23T22:00:10.706 INFO:teuthology.orchestra.run.smithi150.stderr: 92.8% -- replaced with /var/log/ceph/cluster2-client.mirror.46242.log.gz 2024-04-23T22:00:10.708 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.61964.log.gz 2024-04-23T22:00:10.709 INFO:teuthology.orchestra.run.smithi150.stderr: 93.2% -- replaced with /var/log/ceph/cluster2-client.mirror.57771.log.gz 2024-04-23T22:00:10.713 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88428.log 2024-04-23T22:00:10.713 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105554.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63740.log 2024-04-23T22:00:10.713 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88428.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.88428.log.gz 2024-04-23T22:00:10.713 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97466.log 2024-04-23T22:00:10.720 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.63740.log: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.95833.log.gz 2024-04-23T22:00:10.733 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97207.log 2024-04-23T22:00:10.736 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97466.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.97466.log.gz 2024-04-23T22:00:10.743 INFO:teuthology.orchestra.run.smithi150.stderr: 93.3% -- replaced with /var/log/ceph/cluster2-client.mirror.57900.log.gz 2024-04-23T22:00:10.746 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.53087.log.gz 2024-04-23T22:00:10.750 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74926.log 2024-04-23T22:00:10.765 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97207.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.97207.log.gz 2024-04-23T22:00:10.767 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% -- replaced with /var/log/ceph/cluster1-client.mirror.57791.log.gz 2024-04-23T22:00:10.778 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/cluster1-client.mirror.2.106511.log.gz 2024-04-23T22:00:10.781 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97165.log 2024-04-23T22:00:10.781 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74926.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84845.log 2024-04-23T22:00:10.781 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.74926.log.gz 2024-04-23T22:00:10.781 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97165.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34899.log 2024-04-23T22:00:10.783 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.84845.log: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84845.log.gz 2024-04-23T22:00:10.784 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43217.log 2024-04-23T22:00:10.784 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34899.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34899.log.gz 2024-04-23T22:00:10.784 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42625.log 2024-04-23T22:00:10.784 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.43217.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61665.log 2024-04-23T22:00:10.785 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.42625.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95120.log 2024-04-23T22:00:10.785 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.61665.log: 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.97165.log.gz 2024-04-23T22:00:10.788 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96163.log 2024-04-23T22:00:10.788 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95120.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.95120.log.gz 2024-04-23T22:00:10.791 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90610.log 2024-04-23T22:00:10.791 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.96163.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.96163.log.gz 2024-04-23T22:00:10.791 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95455.log 2024-04-23T22:00:10.791 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90610.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.90610.log.gz 2024-04-23T22:00:10.791 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87026.log 2024-04-23T22:00:10.791 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95455.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.95455.log.gz 2024-04-23T22:00:10.793 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.43217.log.gz 2024-04-23T22:00:10.793 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.42625.log.gz 2024-04-23T22:00:10.798 INFO:teuthology.orchestra.run.smithi150.stderr: 92.9% -- replaced with /var/log/ceph/cluster2-client.mirror.3.35645.log.gz 2024-04-23T22:00:10.799 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101506.log 2024-04-23T22:00:10.809 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.87026.log: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.68825.log.gz 2024-04-23T22:00:10.810 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76005.log 2024-04-23T22:00:10.811 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101506.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.101506.log.gz 2024-04-23T22:00:10.818 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% -- replaced with /var/log/ceph/cluster2-client.mirror.2.35627.log.gz 2024-04-23T22:00:10.823 INFO:teuthology.orchestra.run.smithi150.stderr: 91.9% -- replaced with /var/log/ceph/cluster1-client.mirror.87026.log.gz 2024-04-23T22:00:10.823 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97551.log 2024-04-23T22:00:10.823 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76005.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.76005.log.gz 2024-04-23T22:00:10.832 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66343.log 2024-04-23T22:00:10.837 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97551.log: 88.2%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52446.log 2024-04-23T22:00:10.837 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66343.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.66343.log.gz 2024-04-23T22:00:10.837 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42961.log 2024-04-23T22:00:10.837 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52446.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50526.log 2024-04-23T22:00:10.838 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.42961.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51189.log 2024-04-23T22:00:10.838 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50526.log: -- replaced with /var/log/ceph/cluster2-client.mirror.97551.log.gz 2024-04-23T22:00:10.838 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99161.log 2024-04-23T22:00:10.846 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.51189.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57556.log 2024-04-23T22:00:10.847 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99161.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.42961.log.gz 2024-04-23T22:00:10.850 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.51189.log.gz 2024-04-23T22:00:10.850 INFO:teuthology.orchestra.run.smithi150.stderr: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.99161.log.gz 2024-04-23T22:00:10.850 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48246.log 2024-04-23T22:00:10.855 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57556.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101168.log 2024-04-23T22:00:10.855 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48246.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47082.log 2024-04-23T22:00:10.855 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101168.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101168.log.gz 2024-04-23T22:00:10.859 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68976.log 2024-04-23T22:00:10.859 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.47082.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49550.log 2024-04-23T22:00:10.860 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68976.log: 87.7% -- replaced with /var/log/ceph/cluster2-client.mirror.68976.log.gz 2024-04-23T22:00:10.863 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66737.log 2024-04-23T22:00:10.863 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.49550.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79433.log 2024-04-23T22:00:10.864 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66737.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.66737.log.gz 2024-04-23T22:00:10.864 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.47082.log.gz 2024-04-23T22:00:10.865 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52338.log 2024-04-23T22:00:10.866 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.79433.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.79433.log.gz 2024-04-23T22:00:10.866 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87048.log 2024-04-23T22:00:10.867 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52338.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80340.log 2024-04-23T22:00:10.879 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87048.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46305.log 2024-04-23T22:00:10.879 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.80340.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.80340.log.gz 2024-04-23T22:00:10.879 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.90687.log 2024-04-23T22:00:10.898 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.46305.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96904.log 2024-04-23T22:00:10.900 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.90687.log: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.90687.log.gz 2024-04-23T22:00:10.901 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.71062.log.gz 2024-04-23T22:00:10.905 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46327.log 2024-04-23T22:00:10.906 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.96904.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.96904.log.gz 2024-04-23T22:00:10.915 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42040.log 2024-04-23T22:00:10.915 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46327.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40294.log 2024-04-23T22:00:10.915 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.42040.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104433.log 2024-04-23T22:00:10.915 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40294.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103297.log 2024-04-23T22:00:10.926 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36087.log 2024-04-23T22:00:10.926 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104433.log: /var/log/ceph/cluster2-client.mirror.103297.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.104433.log.gz 2024-04-23T22:00:10.926 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.42040.log.gz 2024-04-23T22:00:10.926 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57091.log 2024-04-23T22:00:10.926 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.36087.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65507.log 2024-04-23T22:00:10.927 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57091.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98575.log 2024-04-23T22:00:10.930 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.65507.log: 94.2% -- replaced with /var/log/ceph/cluster2-client.mirror.56767.log.gz 2024-04-23T22:00:10.931 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99853.log 2024-04-23T22:00:10.932 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98575.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.98575.log.gz 2024-04-23T22:00:10.934 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.48246.log.gz 2024-04-23T22:00:10.938 INFO:teuthology.orchestra.run.smithi150.stderr: 54.5% -- replaced with /var/log/ceph/cluster1-client.mirror.36087.log.gz 2024-04-23T22:00:10.939 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.40294.log.gz 2024-04-23T22:00:10.943 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99357.log 2024-04-23T22:00:10.944 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99853.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.99853.log.gz 2024-04-23T22:00:10.957 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105739.log 2024-04-23T22:00:10.959 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99357.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.99357.log.gz 2024-04-23T22:00:10.960 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99760.log 2024-04-23T22:00:10.960 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105739.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.105739.log.gz 2024-04-23T22:00:10.961 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.57091.log.gz 2024-04-23T22:00:10.976 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96020.log 2024-04-23T22:00:10.977 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99760.log: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.99760.log.gz 2024-04-23T22:00:10.989 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95155.log 2024-04-23T22:00:10.996 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.96020.log: 95.9% -- replaced with /var/log/ceph/cluster2-client.mirror.61522.log.gz 2024-04-23T22:00:11.000 INFO:teuthology.orchestra.run.smithi150.stderr: 97.2% -- replaced with /var/log/ceph/cluster2-client.mirror.96020.log.gz 2024-04-23T22:00:11.001 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105177.log 2024-04-23T22:00:11.001 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95155.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.95155.log.gz 2024-04-23T22:00:11.002 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75351.log 2024-04-23T22:00:11.004 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105177.log: 91.4% -- replaced with /var/log/ceph/cluster2-client.mirror.105177.log.gz 2024-04-23T22:00:11.005 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98391.log 2024-04-23T22:00:11.005 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75351.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.75351.log.gz 2024-04-23T22:00:11.005 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90709.log 2024-04-23T22:00:11.006 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98391.log: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.98391.log.gz 2024-04-23T22:00:11.007 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68037.log 2024-04-23T22:00:11.020 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90709.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106685.log 2024-04-23T22:00:11.021 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68037.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.68037.log.gz 2024-04-23T22:00:11.034 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101978.log 2024-04-23T22:00:11.035 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.106685.log: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.106685.log.gz 2024-04-23T22:00:11.035 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88056.log 2024-04-23T22:00:11.036 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101978.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.101978.log.gz 2024-04-23T22:00:11.051 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98141.log 2024-04-23T22:00:11.051 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88056.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.88056.log.gz 2024-04-23T22:00:11.063 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62774.log 2024-04-23T22:00:11.063 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98141.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.98141.log.gz 2024-04-23T22:00:11.070 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.65507.log.gz 2024-04-23T22:00:11.073 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72396.log 2024-04-23T22:00:11.073 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.62774.log: 84.3% -- replaced with /var/log/ceph/cluster2-client.mirror.62774.log.gz 2024-04-23T22:00:11.073 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.104228.log 2024-04-23T22:00:11.086 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.72396.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.44481.log 2024-04-23T22:00:11.087 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.104228.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.104228.log.gz 2024-04-23T22:00:11.100 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90320.log 2024-04-23T22:00:11.113 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.44481.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67700.log 2024-04-23T22:00:11.120 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90320.log: 87.8% 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.61665.log.gz 2024-04-23T22:00:11.121 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41493.log 2024-04-23T22:00:11.122 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.67700.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.67700.log.gz 2024-04-23T22:00:11.123 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41687.log 2024-04-23T22:00:11.123 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41493.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102338.log 2024-04-23T22:00:11.125 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41687.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42331.log 2024-04-23T22:00:11.125 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102338.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102338.log.gz 2024-04-23T22:00:11.126 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98960.log 2024-04-23T22:00:11.126 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.42331.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46500.log 2024-04-23T22:00:11.127 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98960.log: -- replaced with /var/log/ceph/cluster2-client.mirror.90320.log.gz 2024-04-23T22:00:11.127 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.44481.log.gz 2024-04-23T22:00:11.128 INFO:teuthology.orchestra.run.smithi150.stderr: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.98960.log.gz 2024-04-23T22:00:11.131 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57321.log 2024-04-23T22:00:11.132 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46500.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97666.log 2024-04-23T22:00:11.135 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57321.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85592.log 2024-04-23T22:00:11.139 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97666.log: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.97666.log.gz 2024-04-23T22:00:11.139 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35109.log 2024-04-23T22:00:11.141 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85592.log: 91.7% -- replaced with /var/log/ceph/cluster2-client.mirror.85592.log.gz 2024-04-23T22:00:11.144 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82388.log 2024-04-23T22:00:11.148 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.35109.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35109.log.gz 2024-04-23T22:00:11.148 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48863.log 2024-04-23T22:00:11.150 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.82388.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.82388.log.gz 2024-04-23T22:00:11.150 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44970.log 2024-04-23T22:00:11.150 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48863.log: gzip -5 --verbose 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.41493.log.gz 2024-04-23T22:00:11.150 INFO:teuthology.orchestra.run.smithi150.stderr: -- /var/log/ceph/cluster2-client.mirror.37409.log 2024-04-23T22:00:11.151 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44970.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.41687.log.gz 2024-04-23T22:00:11.151 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.42331.log.gz 2024-04-23T22:00:11.152 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75582.log 2024-04-23T22:00:11.156 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.37409.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.44970.log.gz 2024-04-23T22:00:11.156 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.91837.log 2024-04-23T22:00:11.157 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75582.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.75582.log.gz 2024-04-23T22:00:11.157 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47761.log 2024-04-23T22:00:11.158 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.48863.log.gz 2024-04-23T22:00:11.161 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.91837.log: 87.9% -- replaced with /var/log/ceph/cluster1-client.mirror.91837.log.gz 2024-04-23T22:00:11.161 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.78273.log 2024-04-23T22:00:11.164 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.47761.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.78087.log 2024-04-23T22:00:11.166 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.78273.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.78273.log.gz 2024-04-23T22:00:11.166 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72039.log 2024-04-23T22:00:11.169 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.78087.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.78087.log.gz 2024-04-23T22:00:11.170 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86981.log 2024-04-23T22:00:11.173 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.72039.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.72039.log.gz 2024-04-23T22:00:11.174 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55615.log 2024-04-23T22:00:11.175 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.86981.log: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86981.log.gz 2024-04-23T22:00:11.175 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43993.log 2024-04-23T22:00:11.178 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.55615.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54138.log 2024-04-23T22:00:11.182 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.43993.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60322.log 2024-04-23T22:00:11.183 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.54138.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43023.log 2024-04-23T22:00:11.184 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60322.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.103148.log 2024-04-23T22:00:11.187 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.43023.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47699.log 2024-04-23T22:00:11.187 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103148.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.103148.log.gz 2024-04-23T22:00:11.187 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51011.log 2024-04-23T22:00:11.188 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.37409.log.gz 2024-04-23T22:00:11.193 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.90709.log.gz 2024-04-23T22:00:11.193 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.47699.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57212.log 2024-04-23T22:00:11.193 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.51011.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.36414.log 2024-04-23T22:00:11.194 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57212.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53718.log 2024-04-23T22:00:11.194 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.36414.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.93557.log 2024-04-23T22:00:11.194 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.53718.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69259.log 2024-04-23T22:00:11.195 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.93557.log: 92.1% -- replaced with /var/log/ceph/cluster1-client.mirror.93557.log.gz 2024-04-23T22:00:11.195 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106603.log 2024-04-23T22:00:11.196 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69259.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.69259.log.gz 2024-04-23T22:00:11.197 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.49622.log 2024-04-23T22:00:11.197 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.43023.log.gz 2024-04-23T22:00:11.198 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.106603.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.106603.log.gz 2024-04-23T22:00:11.198 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.43993.log.gz 2024-04-23T22:00:11.200 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61723.log 2024-04-23T22:00:11.200 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.49622.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67105.log 2024-04-23T22:00:11.200 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.61723.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34185.log 2024-04-23T22:00:11.201 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.67105.log: 94.1% -- replaced with /var/log/ceph/cluster1-client.mirror.67105.log.gz 2024-04-23T22:00:11.202 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56718.log 2024-04-23T22:00:11.203 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34185.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34185.log.gz 2024-04-23T22:00:11.203 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster1-client.admin.49622.log.gz 2024-04-23T22:00:11.204 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.47699.log.gz 2024-04-23T22:00:11.204 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.47761.log.gz 2024-04-23T22:00:11.205 INFO:teuthology.orchestra.run.smithi150.stderr: 54.2% -- replaced with /var/log/ceph/cluster2-client.mirror.36414.log.gz 2024-04-23T22:00:11.206 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83515.log 2024-04-23T22:00:11.208 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56718.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106121.log 2024-04-23T22:00:11.210 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.83515.log: 95.4% -- replaced with /var/log/ceph/cluster2-client.mirror.105554.log.gz 2024-04-23T22:00:11.212 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100044.log 2024-04-23T22:00:11.212 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106121.log: 94.5% -- replaced with /var/log/ceph/cluster1-client.mirror.83515.log.gz 2024-04-23T22:00:11.212 INFO:teuthology.orchestra.run.smithi150.stderr: 87.7% -- replaced with /var/log/ceph/cluster2-client.mirror.106121.log.gz 2024-04-23T22:00:11.212 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84672.log 2024-04-23T22:00:11.212 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106419.log 2024-04-23T22:00:11.213 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100044.log: 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.100044.log.gz 2024-04-23T22:00:11.213 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34605.log 2024-04-23T22:00:11.214 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106419.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.106419.log.gz 2024-04-23T22:00:11.217 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58646.log 2024-04-23T22:00:11.217 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34605.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34605.log.gz 2024-04-23T22:00:11.217 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.84672.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76734.log 2024-04-23T22:00:11.217 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.84672.log.gz 2024-04-23T22:00:11.217 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58646.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54383.log 2024-04-23T22:00:11.219 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76734.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.76734.log.gz 2024-04-23T22:00:11.219 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67947.log 2024-04-23T22:00:11.220 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.54383.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101798.log 2024-04-23T22:00:11.220 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5/var/log/ceph/cluster1-client.mirror.67947.log: --verbose -- /var/log/ceph/cluster1-client.mirror.103542.log 2024-04-23T22:00:11.223 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101798.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101798.log.gz 2024-04-23T22:00:11.223 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76495.log 2024-04-23T22:00:11.223 INFO:teuthology.orchestra.run.smithi150.stderr: 91.8% -- replaced with /var/log/ceph/cluster1-client.mirror.67947.log.gz 2024-04-23T22:00:11.223 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103542.log: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.51011.log.gz 2024-04-23T22:00:11.223 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.103542.log.gz 2024-04-23T22:00:11.224 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79659.log 2024-04-23T22:00:11.224 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76495.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42432.log 2024-04-23T22:00:11.224 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.79659.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.79659.log.gz 2024-04-23T22:00:11.226 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91232.log 2024-04-23T22:00:11.226 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.42432.log: 92.1% -- replaced with /var/log/ceph/cluster2-client.mirror.76495.log.gz 2024-04-23T22:00:11.226 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45810.log 2024-04-23T22:00:11.226 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.37114.log 2024-04-23T22:00:11.226 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58912.log 2024-04-23T22:00:11.227 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49847.log 2024-04-23T22:00:11.227 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.37114.log: /var/log/ceph/cluster2-client.mirror.45810.log: /var/log/ceph/cluster2-client.mirror.58912.log: /var/log/ceph/cluster2-client.mirror.91232.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.91232.log.gz 2024-04-23T22:00:11.232 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5/var/log/ceph/cluster2-client.mirror.49847.log: --verbose -- /var/log/ceph/cluster2-client.mirror.36329.log 88.7% 2024-04-23T22:00:11.232 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.54138.log.gz 2024-04-23T22:00:11.232 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97489.log 2024-04-23T22:00:11.232 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.36329.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89989.log 2024-04-23T22:00:11.232 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97489.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.107148.log 2024-04-23T22:00:11.232 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89989.log: 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.97489.log.gz 2024-04-23T22:00:11.232 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99731.log 2024-04-23T22:00:11.232 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.107148.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.107148.log.gz 2024-04-23T22:00:11.232 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81596.log 2024-04-23T22:00:11.235 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99731.log: 96.5% -- replaced with /var/log/ceph/cluster2-client.mirror.46327.log.gz 2024-04-23T22:00:11.235 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.53718.log.gz 2024-04-23T22:00:11.237 INFO:teuthology.orchestra.run.smithi150.stderr: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.99731.log.gz 2024-04-23T22:00:11.237 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.54383.log.gz 2024-04-23T22:00:11.238 INFO:teuthology.orchestra.run.smithi150.stderr: 93.2% -- replaced with /var/log/ceph/cluster2-client.mirror.52338.log.gz 2024-04-23T22:00:11.239 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.37778.log 2024-04-23T22:00:11.240 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81596.log: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.50526.log.gz 2024-04-23T22:00:11.242 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.42432.log.gz 2024-04-23T22:00:11.244 INFO:teuthology.orchestra.run.smithi150.stderr: 84.5% 84.5% -- replaced with /var/log/ceph/cluster2-client.mirror.58912.log.gz 2024-04-23T22:00:11.244 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.58646.log.gz 2024-04-23T22:00:11.244 INFO:teuthology.orchestra.run.smithi150.stderr: 92.6% -- replaced with /var/log/ceph/cluster2-client.mirror.81596.log.gz 2024-04-23T22:00:11.244 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81726.log 2024-04-23T22:00:11.245 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.37778.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73171.log 2024-04-23T22:00:11.245 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81726.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.81726.log.gz 2024-04-23T22:00:11.247 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102968.log 2024-04-23T22:00:11.247 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.56718.log.gz 2024-04-23T22:00:11.249 INFO:teuthology.orchestra.run.smithi148.stderr: 91.4% -- replaced with /var/log/ceph/cluster2-mgr.x.log.gz 2024-04-23T22:00:11.249 INFO:teuthology.orchestra.run.smithi150.stderr: 54.9% -- replaced with /var/log/ceph/cluster2-client.mirror.36329.log.gz 2024-04-23T22:00:11.250 INFO:teuthology.orchestra.run.smithi150.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.45810.log.gz 2024-04-23T22:00:11.256 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.49847.log.gz 2024-04-23T22:00:11.256 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73171.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.73171.log.gz 2024-04-23T22:00:11.256 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91862.log 2024-04-23T22:00:11.256 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102968.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102968.log.gz 2024-04-23T22:00:11.256 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68215.log 2024-04-23T22:00:11.258 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.91862.log: 91.8% -- replaced with /var/log/ceph/cluster2-client.mirror.91862.log.gz 2024-04-23T22:00:11.258 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101123.log 2024-04-23T22:00:11.259 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.87048.log.gz 2024-04-23T22:00:11.260 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68215.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.68215.log.gz 2024-04-23T22:00:11.267 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83863.log 2024-04-23T22:00:11.268 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101123.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100492.log 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101123.log.gz 2024-04-23T22:00:11.268 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:11.268 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40038.log 2024-04-23T22:00:11.268 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100492.log: /var/log/ceph/cluster2-client.mirror.83863.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.83863.log.gz 2024-04-23T22:00:11.268 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49350.log 2024-04-23T22:00:11.268 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40038.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106338.log 2024-04-23T22:00:11.268 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.49350.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.100492.log.gz 2024-04-23T22:00:11.268 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91805.log 2024-04-23T22:00:11.268 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38389.log 2024-04-23T22:00:11.268 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.91805.log: /var/log/ceph/cluster2-client.mirror.106338.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.91805.log.gz 2024-04-23T22:00:11.268 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.106338.log.gz 2024-04-23T22:00:11.269 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73299.log 2024-04-23T22:00:11.272 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.38389.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34393.log 2024-04-23T22:00:11.272 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73299.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.73299.log.gz 2024-04-23T22:00:11.272 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53122.log 2024-04-23T22:00:11.272 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34393.log: gzip 0.0% -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67482.log 2024-04-23T22:00:11.272 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.admin.34393.log.gz 2024-04-23T22:00:11.272 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.53122.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42172.log 2024-04-23T22:00:11.272 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.40038.log.gz 2024-04-23T22:00:11.276 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.67482.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.67482.log.gz 2024-04-23T22:00:11.276 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69938.log 2024-04-23T22:00:11.276 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.42172.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44873.log 2024-04-23T22:00:11.276 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69938.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49527.log 2024-04-23T22:00:11.276 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44873.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43411.log 2024-04-23T22:00:11.279 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.49527.log: 92.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69938.log.gz 2024-04-23T22:00:11.279 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101911.log 2024-04-23T22:00:11.279 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.43411.log: 92.7% -- replaced with /var/log/ceph/cluster1-client.mirror.57212.log.gz 2024-04-23T22:00:11.279 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.42172.log.gz 2024-04-23T22:00:11.279 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.63740.log.gz 2024-04-23T22:00:11.279 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.37778.log.gz 2024-04-23T22:00:11.279 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.38389.log.gz 2024-04-23T22:00:11.281 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.36397.log 2024-04-23T22:00:11.281 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster1-client.mirror.101911.log: -- /var/log/ceph/cluster1-client.mirror.101258.log 2024-04-23T22:00:11.281 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101911.log.gz 2024-04-23T22:00:11.283 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.36397.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57576.log 2024-04-23T22:00:11.283 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101258.log: 86.7% 57.1% -- replaced with /var/log/ceph/cluster2-client.mirror.36397.log.gz 2024-04-23T22:00:11.284 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.101258.log.gz 2024-04-23T22:00:11.348 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69881.log 2024-04-23T22:00:11.348 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57576.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100299.log 2024-04-23T22:00:11.348 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69881.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85993.log 2024-04-23T22:00:11.349 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.69881.log.gz 2024-04-23T22:00:11.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100299.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76672.log 2024-04-23T22:00:11.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85993.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46694.log 2024-04-23T22:00:11.349 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.85993.log.gz 2024-04-23T22:00:11.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76672.log: gzip -5 --verbose -- 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.76672.log.gz 2024-04-23T22:00:11.349 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster2-client.mirror.75470.log 2024-04-23T22:00:11.349 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.44873.log.gz 2024-04-23T22:00:11.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46694.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103511.log 2024-04-23T22:00:11.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75470.log: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.53122.log.gz 90.5% 2024-04-23T22:00:11.349 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.100299.log.gz 2024-04-23T22:00:11.349 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.75470.log.gz 2024-04-23T22:00:11.349 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34731.log 2024-04-23T22:00:11.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.103511.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.103946.log 2024-04-23T22:00:11.350 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34731.log: 0.0%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100157.log 2024-04-23T22:00:11.350 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.admin.34731.log.gz 2024-04-23T22:00:11.350 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.74186.log 2024-04-23T22:00:11.350 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103946.log: /var/log/ceph/cluster2-client.mirror.100157.log: 91.2% -- replaced with /var/log/ceph/cluster2-client.mirror.103511.log.gz 2024-04-23T22:00:11.350 INFO:teuthology.orchestra.run.smithi150.stderr: 86.8% -- replaced with /var/log/ceph/cluster1-client.mirror.103946.log.gz 2024-04-23T22:00:11.350 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92307.log 2024-04-23T22:00:11.350 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.74186.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53017.log 2024-04-23T22:00:11.350 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.92307.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.92307.log.gz 2024-04-23T22:00:11.350 INFO:teuthology.orchestra.run.smithi150.stderr: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.100157.log.gz 2024-04-23T22:00:11.350 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41590.log 2024-04-23T22:00:11.350 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.53017.log: 91.9% -- replaced with /var/log/ceph/cluster1-client.mirror.74186.log.gz 2024-04-23T22:00:11.350 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72245.log 2024-04-23T22:00:11.351 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41590.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65373.log 2024-04-23T22:00:11.351 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69846.log 2024-04-23T22:00:11.351 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.72245.log: /var/log/ceph/cluster2-client.mirror.65373.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.72245.log.gz 2024-04-23T22:00:11.351 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90150.log 2024-04-23T22:00:11.351 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.65373.log.gz 2024-04-23T22:00:11.351 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67021.log 2024-04-23T22:00:11.351 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69846.log: /var/log/ceph/cluster2-client.mirror.90150.log: 87.9% 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.90150.log.gz 2024-04-23T22:00:11.351 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.69846.log.gz 2024-04-23T22:00:11.351 INFO:teuthology.orchestra.run.smithi150.stderr: 92.3% -- replaced with /var/log/ceph/cluster2-client.mirror.1.35591.log.gz 2024-04-23T22:00:11.351 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95264.log 2024-04-23T22:00:11.351 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/cluster2-client.mirror.67021.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99451.log 2024-04-23T22:00:11.351 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95264.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.67021.log.gz 2024-04-23T22:00:11.352 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.3.35548.log 2024-04-23T22:00:11.352 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99451.log: 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.99451.log.gz 2024-04-23T22:00:11.352 INFO:teuthology.orchestra.run.smithi150.stderr: 91.6%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97709.log 2024-04-23T22:00:11.352 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.95264.log.gz 2024-04-23T22:00:11.352 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.3.35548.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55503.log 2024-04-23T22:00:11.352 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97709.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35645.log 2024-04-23T22:00:11.352 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55503.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60767.log 2024-04-23T22:00:11.352 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.35645.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.3.61165.log 2024-04-23T22:00:11.352 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60767.log: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.60767.log.gz 2024-04-23T22:00:11.352 INFO:teuthology.orchestra.run.smithi150.stderr: 90.6% -- replaced with /var/log/ceph/cluster2-client.mirror.97709.log.gz 2024-04-23T22:00:11.352 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100115.log 2024-04-23T22:00:11.352 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/cluster2-client.mirror.3.61165.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52119.log 2024-04-23T22:00:11.352 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99096.log 2024-04-23T22:00:11.353 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52119.log: /var/log/ceph/cluster2-client.mirror.100115.log: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.100115.log.gz 2024-04-23T22:00:11.353 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102248.log 2024-04-23T22:00:11.353 INFO:teuthology.orchestra.run.smithi150.stderr:gzip 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.89989.log.gz 2024-04-23T22:00:11.353 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.78188.log 2024-04-23T22:00:11.353 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102248.log: /var/log/ceph/cluster2-client.mirror.99096.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.102248.log.gz 2024-04-23T22:00:11.353 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48537.log 2024-04-23T22:00:11.353 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.78188.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52165.log 2024-04-23T22:00:11.353 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48537.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.78188.log.gz 2024-04-23T22:00:11.353 INFO:teuthology.orchestra.run.smithi150.stderr: 90.6%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55328.log 2024-04-23T22:00:11.353 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.99096.log.gz/var/log/ceph/cluster2-client.mirror.52165.log: 2024-04-23T22:00:11.353 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.104684.log 2024-04-23T22:00:11.353 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55328.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76159.log 2024-04-23T22:00:11.354 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95699.log 2024-04-23T22:00:11.354 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76159.log: /var/log/ceph/cluster1-client.mirror.104684.log: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.104684.log.gz 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.76159.log.gz 2024-04-23T22:00:11.354 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:11.354 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52401.log 2024-04-23T22:00:11.354 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95699.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42137.log 2024-04-23T22:00:11.354 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.95699.log.gz 2024-04-23T22:00:11.354 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100257.log 2024-04-23T22:00:11.354 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.52401.log: /var/log/ceph/cluster2-client.mirror.42137.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73709.log 2024-04-23T22:00:11.354 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100257.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98342.log 2024-04-23T22:00:11.354 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73709.log: 89.2% 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.73709.log.gz 2024-04-23T22:00:11.354 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.100257.log.gz 2024-04-23T22:00:11.354 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36053.log 2024-04-23T22:00:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5/var/log/ceph/cluster2-client.mirror.98342.log: --verbose -- /var/log/ceph/cluster2-client.mirror.75687.log 2024-04-23T22:00:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.36053.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.98342.log.gz 2024-04-23T22:00:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98504.log 2024-04-23T22:00:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75687.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72116.log 2024-04-23T22:00:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98504.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76637.log 2024-04-23T22:00:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.72116.log: 92.2% -- replaced with /var/log/ceph/cluster2-client.mirror.75687.log.gz 2024-04-23T22:00:11.355 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.72116.log.gz 2024-04-23T22:00:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65852.log 2024-04-23T22:00:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76637.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.98504.log.gz 2024-04-23T22:00:11.355 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.76637.log.gzgzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38121.log 2024-04-23T22:00:11.355 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/cluster2-client.mirror.65852.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106264.log 2024-04-23T22:00:11.355 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.38121.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49269.log 2024-04-23T22:00:11.356 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.65852.log.gz 2024-04-23T22:00:11.356 INFO:teuthology.orchestra.run.smithi150.stderr:gzip/var/log/ceph/cluster2-client.mirror.106264.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94924.log 2024-04-23T22:00:11.356 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.49269.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.104389.log 2024-04-23T22:00:11.356 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58376.log 2024-04-23T22:00:11.356 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.104389.log: 91.2% -- replaced with /var/log/ceph/cluster2-client.mirror.106264.log.gz 2024-04-23T22:00:11.356 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79398.log 2024-04-23T22:00:11.356 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.104389.log.gz/var/log/ceph/cluster2-client.mirror.58376.log: 2024-04-23T22:00:11.356 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94924.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66580.log 2024-04-23T22:00:11.356 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9%/var/log/ceph/cluster2-client.mirror.79398.log: -- replaced with /var/log/ceph/cluster2-client.mirror.94924.log.gz 2024-04-23T22:00:11.356 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.79398.log.gz 2024-04-23T22:00:11.356 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105506.log 2024-04-23T22:00:11.356 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85634.log 2024-04-23T22:00:11.356 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66580.log: /var/log/ceph/cluster2-client.mirror.105506.log: gzip -5 --verbose -- 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.66580.log.gz 2024-04-23T22:00:11.357 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster2-client.mirror.56237.log 2024-04-23T22:00:11.357 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.105506.log.gz 2024-04-23T22:00:11.357 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster2-client.mirror.85634.log: -- /var/log/ceph/cluster2-client.admin.34644.log 2024-04-23T22:00:11.357 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51046.log 2024-04-23T22:00:11.357 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56237.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.42137.log.gz 2024-04-23T22:00:11.357 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34644.log: 93.8% -- replaced with /var/log/ceph/cluster2-client.mirror.3.61165.log.gz 2024-04-23T22:00:11.357 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34644.log.gz 2024-04-23T22:00:11.357 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106143.log 2024-04-23T22:00:11.357 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.43411.log.gz 2024-04-23T22:00:11.357 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% -- replaced with /var/log/ceph/cluster2-client.mirror.57556.log.gz 2024-04-23T22:00:11.357 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.51046.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91904.log 2024-04-23T22:00:11.357 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106143.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44813.log 2024-04-23T22:00:11.358 INFO:teuthology.orchestra.run.smithi150.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.106143.log.gz 2024-04-23T22:00:11.358 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.91904.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.3.62975.log 2024-04-23T22:00:11.358 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44813.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.51681.log 2024-04-23T22:00:11.358 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster1-client.mirror.3.62975.log: -- /var/log/ceph/cluster1-client.mirror.101101.log 2024-04-23T22:00:11.358 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.51681.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.56237.log.gz 2024-04-23T22:00:11.358 INFO:teuthology.orchestra.run.smithi150.stderr: 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.58376.log.gz 2024-04-23T22:00:11.358 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.49591.log 2024-04-23T22:00:11.358 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101101.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.38121.log.gz 2024-04-23T22:00:11.358 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/cluster1-client.admin.51681.log.gz 2024-04-23T22:00:11.358 INFO:teuthology.orchestra.run.smithi150.stderr: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.101101.log.gz 2024-04-23T22:00:11.358 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95811.log 2024-04-23T22:00:11.358 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.49591.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97288.log 2024-04-23T22:00:11.359 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.95811.log: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.55328.log.gz 2024-04-23T22:00:11.359 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.46694.log.gz 2024-04-23T22:00:11.359 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.48537.log.gz 2024-04-23T22:00:11.359 INFO:teuthology.orchestra.run.smithi150.stderr: 81.2% -- replaced with /var/log/ceph/cluster1-client.mirror.35645.log.gz 2024-04-23T22:00:11.359 INFO:teuthology.orchestra.run.smithi150.stderr: 91.9% -- replaced with /var/log/ceph/cluster1-client.mirror.95811.log.gz 2024-04-23T22:00:11.359 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38727.log 2024-04-23T22:00:11.359 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97288.log: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.97288.log.gz 2024-04-23T22:00:11.359 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81108.log 2024-04-23T22:00:11.359 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.38727.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34518.log 2024-04-23T22:00:11.359 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81108.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.81108.log.gz 2024-04-23T22:00:11.359 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100867.log 2024-04-23T22:00:11.359 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34518.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34518.log.gz 2024-04-23T22:00:11.359 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.39192.log 2024-04-23T22:00:11.359 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100867.log: 90.4% -- replaced with /var/log/ceph/cluster2-client.mirror.100867.log.gz 2024-04-23T22:00:11.360 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62274.log 2024-04-23T22:00:11.360 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.39192.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79304.log 2024-04-23T22:00:11.360 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.62274.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57278.log 2024-04-23T22:00:11.362 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.79304.log: 92.2% -- replaced with /var/log/ceph/cluster2-client.mirror.79304.log.gz 2024-04-23T22:00:11.363 INFO:teuthology.orchestra.run.smithi150.stderr: 87.7% -- replaced with /var/log/ceph/cluster1-client.admin.37114.log.gz 2024-04-23T22:00:11.367 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81220.log 2024-04-23T22:00:11.373 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57278.log: 64.3% -- replaced with /var/log/ceph/cluster1-client.mirror.36053.log.gz 2024-04-23T22:00:11.374 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.41590.log.gz 2024-04-23T22:00:11.375 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.103969.log 2024-04-23T22:00:11.375 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81220.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.81220.log.gz 2024-04-23T22:00:11.376 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.72396.log.gz 2024-04-23T22:00:11.379 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.46262.log 2024-04-23T22:00:11.379 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103969.log: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.103969.log.gz 2024-04-23T22:00:11.379 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85365.log 2024-04-23T22:00:11.381 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.46262.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51224.log 2024-04-23T22:00:11.381 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85365.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.85365.log.gz 2024-04-23T22:00:11.381 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.0.36207.log 2024-04-23T22:00:11.383 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.51224.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87006.log 2024-04-23T22:00:11.383 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.0.36207.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58545.log 2024-04-23T22:00:11.384 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87006.log: 93.2% -- replaced with /var/log/ceph/cluster1-client.mirror.55615.log.gz 2024-04-23T22:00:11.384 INFO:teuthology.orchestra.run.smithi150.stderr: 92.1% -- replaced with /var/log/ceph/cluster2-client.mirror.87006.log.gz 2024-04-23T22:00:11.385 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.85634.log.gz 2024-04-23T22:00:11.390 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.37374.log 2024-04-23T22:00:11.391 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58545.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.45470.log 2024-04-23T22:00:11.391 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.37374.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103689.log 2024-04-23T22:00:11.391 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster1-client.mirror.45470.log: -- /var/log/ceph/cluster2-client.mirror.71020.log 2024-04-23T22:00:11.391 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72329.log 2024-04-23T22:00:11.391 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71020.log: /var/log/ceph/cluster2-client.mirror.103689.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.103689.log.gz 2024-04-23T22:00:11.391 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98271.log 2024-04-23T22:00:11.391 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.72329.log: 91.9% -- replaced with /var/log/ceph/cluster2-client.mirror.71020.log.gz 2024-04-23T22:00:11.391 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.72329.log.gz 2024-04-23T22:00:11.391 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97401.log 2024-04-23T22:00:11.391 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98271.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.98271.log.gz 2024-04-23T22:00:11.392 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101393.log 2024-04-23T22:00:11.393 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97401.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.97401.log.gz 2024-04-23T22:00:11.396 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74433.log 2024-04-23T22:00:11.396 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101393.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101393.log.gz 2024-04-23T22:00:11.396 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101753.log 2024-04-23T22:00:11.396 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83030.log 2024-04-23T22:00:11.398 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74433.log: /var/log/ceph/cluster1-client.mirror.101753.log: 86.6% 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.74433.log.gz 2024-04-23T22:00:11.398 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.101753.log.gz 2024-04-23T22:00:11.398 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77957.log 2024-04-23T22:00:11.398 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91463.log 2024-04-23T22:00:11.399 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.77957.log: /var/log/ceph/cluster2-client.mirror.83030.log: 91.7% -- replaced with /var/log/ceph/cluster1-client.mirror.77957.log.gz 2024-04-23T22:00:11.400 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.91463.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.91463.log.gz 2024-04-23T22:00:11.402 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90809.log 2024-04-23T22:00:11.403 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.39650.log 2024-04-23T22:00:11.403 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90809.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.90809.log.gz 2024-04-23T22:00:11.403 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.51224.log.gz 2024-04-23T22:00:11.404 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51418.log 2024-04-23T22:00:11.404 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.39650.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.58545.log.gz 2024-04-23T22:00:11.407 INFO:teuthology.orchestra.run.smithi150.stderr: 91.9% -- replaced with /var/log/ceph/cluster2-client.mirror.83030.log.gz 2024-04-23T22:00:11.407 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34391.log 2024-04-23T22:00:11.407 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.51418.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82493.log 2024-04-23T22:00:11.407 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97803.log 2024-04-23T22:00:11.407 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34391.log: /var/log/ceph/cluster2-client.mirror.82493.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34391.log.gz 2024-04-23T22:00:11.412 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose 87.9% -- /var/log/ceph/cluster2-client.mirror.53437.log 2024-04-23T22:00:11.412 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.82493.log.gz 2024-04-23T22:00:11.412 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98249.log 2024-04-23T22:00:11.412 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97803.log: /var/log/ceph/cluster2-client.mirror.53437.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93480.log 2024-04-23T22:00:11.413 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98249.log: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.97803.log.gz 2024-04-23T22:00:11.413 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.39906.log 2024-04-23T22:00:11.413 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.98249.log.gz 2024-04-23T22:00:11.413 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.93480.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99054.log 2024-04-23T22:00:11.413 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9%/var/log/ceph/cluster2-client.mirror.39906.log: -- replaced with /var/log/ceph/cluster2-client.mirror.93480.log.gz 2024-04-23T22:00:11.413 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105644.log 2024-04-23T22:00:11.413 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99054.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97446.log 2024-04-23T22:00:11.413 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105644.log: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.99054.log.gz 2024-04-23T22:00:11.426 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/cluster1-client.admin.49591.log.gz 2024-04-23T22:00:11.426 INFO:teuthology.orchestra.run.smithi150.stderr: 91.3% -- replaced with /var/log/ceph/cluster2-client.mirror.105644.log.gz 2024-04-23T22:00:11.426 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47276.log 2024-04-23T22:00:11.426 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97446.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101191.log 2024-04-23T22:00:11.426 INFO:teuthology.orchestra.run.smithi150.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.97446.log.gz/var/log/ceph/cluster2-client.mirror.47276.log: 2024-04-23T22:00:11.426 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86590.log 2024-04-23T22:00:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101191.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106808.log 2024-04-23T22:00:11.427 INFO:teuthology.orchestra.run.smithi150.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.101191.log.gz 2024-04-23T22:00:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86590.log: /var/log/ceph/cluster1-client.mirror.106808.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106808.log.gz 2024-04-23T22:00:11.427 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.86590.log.gz 2024-04-23T22:00:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102901.log 2024-04-23T22:00:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102316.log 2024-04-23T22:00:11.427 INFO:teuthology.orchestra.run.smithi150.stderr: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.47276.log.gz 2024-04-23T22:00:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102901.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.102901.log.gz 2024-04-23T22:00:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- 88.5% -- replaced with /var/log/ceph/cluster1-client.mirror.51418.log.gz 2024-04-23T22:00:11.427 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster2-client.mirror.70766.log 2024-04-23T22:00:11.427 INFO:teuthology.orchestra.run.smithi150.stderr: 92.8%/var/log/ceph/cluster1-client.mirror.102316.log: -- replaced with /var/log/ceph/cluster2-client.mirror.57321.log.gz 2024-04-23T22:00:11.427 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102316.log.gz 2024-04-23T22:00:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104831.log 2024-04-23T22:00:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.70766.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52423.log 2024-04-23T22:00:11.427 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104831.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.70766.log.gz 2024-04-23T22:00:11.428 INFO:teuthology.orchestra.run.smithi150.stderr:gzip 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.104831.log.gz 2024-04-23T22:00:11.428 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93250.log 2024-04-23T22:00:11.428 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.0.83308.log 2024-04-23T22:00:11.428 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.93250.log: /var/log/ceph/cluster2-client.mirror.52423.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.51046.log.gz 2024-04-23T22:00:11.428 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76769.log 2024-04-23T22:00:11.428 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.93250.log.gz 2024-04-23T22:00:11.428 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.0.83308.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55970.log 2024-04-23T22:00:11.428 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76769.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.76769.log.gz 2024-04-23T22:00:11.428 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74814.log 2024-04-23T22:00:11.428 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55970.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.103565.log 2024-04-23T22:00:11.428 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74814.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.74814.log.gz 2024-04-23T22:00:11.428 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48960.log 2024-04-23T22:00:11.428 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103565.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.103565.log.gz 2024-04-23T22:00:11.430 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.38727.log.gz 2024-04-23T22:00:11.432 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59610.log 2024-04-23T22:00:11.435 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48960.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74468.log 2024-04-23T22:00:11.435 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.59610.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57341.log 2024-04-23T22:00:11.435 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74468.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.74468.log.gz 2024-04-23T22:00:11.435 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88614.log 2024-04-23T22:00:11.435 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57341.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.78376.log 2024-04-23T22:00:11.438 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88614.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.88614.log.gz 2024-04-23T22:00:11.439 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83930.log 2024-04-23T22:00:11.443 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.78376.log: 92.1% -- replaced with /var/log/ceph/cluster2-client.mirror.78376.log.gz 2024-04-23T22:00:11.443 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75729.log 2024-04-23T22:00:11.443 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.83930.log: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.83930.log.gz 2024-04-23T22:00:11.444 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63615.log 2024-04-23T22:00:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75729.log: 94.1% -- replaced with /var/log/ceph/cluster2-client.mirror.75729.log.gz 2024-04-23T22:00:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106958.log 2024-04-23T22:00:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.63615.log: 84.3% -- replaced with /var/log/ceph/cluster2-client.mirror.63615.log.gz 2024-04-23T22:00:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99973.log 2024-04-23T22:00:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106958.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.106958.log.gz 2024-04-23T22:00:11.448 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89103.log 2024-04-23T22:00:11.450 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99973.log: 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.99973.log.gz 2024-04-23T22:00:11.450 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36019.log 2024-04-23T22:00:11.450 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89103.log: 95.5% -- replaced with /var/log/ceph/cluster2-client.mirror.91904.log.gz 2024-04-23T22:00:11.451 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.89103.log.gz 2024-04-23T22:00:11.451 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61941.log 2024-04-23T22:00:11.451 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.36019.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81921.log 2024-04-23T22:00:11.455 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.53017.log.gz 2024-04-23T22:00:11.457 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.61941.log: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.61941.log.gz 2024-04-23T22:00:11.458 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99031.log 2024-04-23T22:00:11.458 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81921.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75197.log 2024-04-23T22:00:11.458 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.81921.log.gz 2024-04-23T22:00:11.458 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99031.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73398.log 2024-04-23T22:00:11.458 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75197.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.75197.log.gz 2024-04-23T22:00:11.458 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.2.104912.log 2024-04-23T22:00:11.458 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73398.log: 90.6% -- replaced with /var/log/ceph/cluster2-client.mirror.99031.log.gz 2024-04-23T22:00:11.458 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65909.log 2024-04-23T22:00:11.459 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.2.104912.log: 92.9% -- replaced with /var/log/ceph/cluster1-client.mirror.46305.log.gz 2024-04-23T22:00:11.460 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% -- replaced with /var/log/ceph/cluster1-client.mirror.2.104912.log.gz 2024-04-23T22:00:11.460 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62487.log 2024-04-23T22:00:11.462 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.65909.log: 92.2% -- replaced with /var/log/ceph/cluster2-client.mirror.65909.log.gz 2024-04-23T22:00:11.462 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87294.log 2024-04-23T22:00:11.466 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.62487.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.52119.log.gz 2024-04-23T22:00:11.467 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.52165.log.gz 2024-04-23T22:00:11.467 INFO:teuthology.orchestra.run.smithi150.stderr: 97.0% -- replaced with /var/log/ceph/cluster2-client.mirror.73398.log.gz 2024-04-23T22:00:11.468 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90270.log 2024-04-23T22:00:11.469 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87294.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.87294.log.gz 2024-04-23T22:00:11.469 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94744.log 2024-04-23T22:00:11.470 INFO:teuthology.orchestra.run.smithi150.stderr: 4.7% -- replaced with /var/log/ceph/cluster1-client.mirror.36019.log.gz 2024-04-23T22:00:11.470 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.37374.log.gz 2024-04-23T22:00:11.471 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.39906.log.gz 2024-04-23T22:00:11.471 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.53437.log.gz 2024-04-23T22:00:11.472 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.39650.log.gz 2024-04-23T22:00:11.473 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90270.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.90270.log.gz 2024-04-23T22:00:11.473 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49462.log 2024-04-23T22:00:11.480 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94744.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.94744.log.gz 2024-04-23T22:00:11.483 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66297.log 2024-04-23T22:00:11.483 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.49462.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44908.log 2024-04-23T22:00:11.483 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66297.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.66297.log.gz 2024-04-23T22:00:11.483 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82528.log 2024-04-23T22:00:11.487 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44908.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101956.log 2024-04-23T22:00:11.487 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.82528.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.82528.log.gz 2024-04-23T22:00:11.488 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101956.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101956.log.gz 2024-04-23T22:00:11.490 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105592.log 2024-04-23T22:00:11.490 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92985.log 2024-04-23T22:00:11.490 INFO:teuthology.orchestra.run.smithi150.stderr: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.49269.log.gz 2024-04-23T22:00:11.490 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.49350.log.gz 2024-04-23T22:00:11.491 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.105592.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.105592.log.gz 2024-04-23T22:00:11.491 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77133.log 2024-04-23T22:00:11.491 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.92985.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.92985.log.gz 2024-04-23T22:00:11.492 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43155.log 2024-04-23T22:00:11.492 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77133.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.77133.log.gz 2024-04-23T22:00:11.493 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95239.log 2024-04-23T22:00:11.493 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.43155.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35903.log 2024-04-23T22:00:11.494 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.95239.log: 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.95239.log.gz 2024-04-23T22:00:11.497 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77582.log 2024-04-23T22:00:11.497 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.35903.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35903.log.gz 2024-04-23T22:00:11.497 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71962.log 2024-04-23T22:00:11.500 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.77582.log: 91.6% -- replaced with /var/log/ceph/cluster1-client.mirror.77582.log.gz 2024-04-23T22:00:11.500 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96781.log 2024-04-23T22:00:11.501 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71962.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.71962.log.gz 2024-04-23T22:00:11.501 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68072.log 2024-04-23T22:00:11.503 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.96781.log: 91.8% -- replaced with /var/log/ceph/cluster1-client.mirror.96781.log.gz 2024-04-23T22:00:11.504 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102091.log 2024-04-23T22:00:11.506 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68072.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.68072.log.gz 2024-04-23T22:00:11.506 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35576.log 2024-04-23T22:00:11.506 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102091.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.44908.log.gz 2024-04-23T22:00:11.506 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102091.log.gz 2024-04-23T22:00:11.506 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90941.log 2024-04-23T22:00:11.507 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.55970.log.gz 2024-04-23T22:00:11.509 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.35576.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89890.log 2024-04-23T22:00:11.509 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90941.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.48960.log.gz 2024-04-23T22:00:11.510 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.90941.log.gz 2024-04-23T22:00:11.510 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94423.log 2024-04-23T22:00:11.510 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89890.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.89890.log.gz 2024-04-23T22:00:11.510 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73264.log 2024-04-23T22:00:11.510 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94423.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.94423.log.gz 2024-04-23T22:00:11.512 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70495.log 2024-04-23T22:00:11.512 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73264.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.73264.log.gz 2024-04-23T22:00:11.514 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.70495.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43737.log 2024-04-23T22:00:11.515 INFO:teuthology.orchestra.run.smithi150.stderr: 91.5% -- replaced with /var/log/ceph/cluster1-client.mirror.70495.log.gz 2024-04-23T22:00:11.517 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.55503.log.gz 2024-04-23T22:00:11.517 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63644.log 2024-04-23T22:00:11.517 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.43737.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104474.log 2024-04-23T22:00:11.522 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.63644.log: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.46500.log.gz 2024-04-23T22:00:11.526 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.3.64180.log 2024-04-23T22:00:11.531 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104474.log: 91.5% -- replaced with /var/log/ceph/cluster2-client.mirror.104474.log.gz 2024-04-23T22:00:11.532 INFO:teuthology.orchestra.run.smithi150.stderr: 92.8% -- replaced with /var/log/ceph/cluster2-client.mirror.39192.log.gz 2024-04-23T22:00:11.534 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51608.log 2024-04-23T22:00:11.535 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.3.64180.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.43155.log.gz 2024-04-23T22:00:11.537 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.43737.log.gz 2024-04-23T22:00:11.544 INFO:teuthology.orchestra.run.smithi150.stderr: 94.0% 92.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57278.log.gz 2024-04-23T22:00:11.549 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92722.log 2024-04-23T22:00:11.550 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.51608.log: 81.5% -- replaced with /var/log/ceph/cluster1-client.mirror.35576.log.gz 2024-04-23T22:00:11.552 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.3.64180.log.gz 2024-04-23T22:00:11.563 INFO:teuthology.orchestra.run.smithi150.stderr: 95.5% -- replaced with /var/log/ceph/cluster2-client.mirror.103297.log.gz 2024-04-23T22:00:11.564 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83005.log 2024-04-23T22:00:11.565 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.92722.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41167.log 2024-04-23T22:00:11.566 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.83005.log: 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.83005.log.gz 2024-04-23T22:00:11.566 INFO:teuthology.orchestra.run.smithi150.stderr: 92.0%gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100966.log 2024-04-23T22:00:11.567 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41167.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98940.log 2024-04-23T22:00:11.570 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.100966.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.100966.log.gz 2024-04-23T22:00:11.571 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56148.log 2024-04-23T22:00:11.572 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98940.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.98940.log.gz 2024-04-23T22:00:11.579 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.92722.log.gz 2024-04-23T22:00:11.580 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58199.log 2024-04-23T22:00:11.581 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56148.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98727.log 2024-04-23T22:00:11.582 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58199.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34645.log 2024-04-23T22:00:11.583 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98727.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.98727.log.gz 2024-04-23T22:00:11.586 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86949.log 2024-04-23T22:00:11.587 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34645.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34645.log.gz 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.41167.log.gz 2024-04-23T22:00:11.587 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:11.587 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99386.log 2024-04-23T22:00:11.588 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86949.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.86949.log.gz 2024-04-23T22:00:11.588 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44549.log 2024-04-23T22:00:11.588 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99386.log: 92.7% -- replaced with /var/log/ceph/cluster1-client.mirror.0.36207.log.gz 2024-04-23T22:00:11.591 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.99386.log.gz 2024-04-23T22:00:11.591 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101641.log 2024-04-23T22:00:11.591 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44549.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51906.log 2024-04-23T22:00:11.591 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101641.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101641.log.gz 2024-04-23T22:00:11.593 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55703.log 2024-04-23T22:00:11.594 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.51906.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73127.log 2024-04-23T22:00:11.597 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55703.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96878.log 2024-04-23T22:00:11.597 INFO:teuthology.orchestra.run.smithi150.stderr: 91.6% -- replaced with /var/log/ceph/cluster1-client.mirror.0.83308.log.gz 2024-04-23T22:00:11.598 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73127.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77979.log 2024-04-23T22:00:11.599 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.96878.log: 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.96878.log.gz 2024-04-23T22:00:11.601 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68129.log 2024-04-23T22:00:11.604 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77979.log: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.56148.log.gz 2024-04-23T22:00:11.613 INFO:teuthology.orchestra.run.smithi150.stderr: 97.0% -- replaced with /var/log/ceph/cluster2-client.mirror.77979.log.gz 2024-04-23T22:00:11.625 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96314.log 2024-04-23T22:00:11.630 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68129.log: 92.2% -- replaced with /var/log/ceph/cluster2-client.mirror.68129.log.gz 2024-04-23T22:00:11.630 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60849.log 2024-04-23T22:00:11.632 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.96314.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.96314.log.gz 2024-04-23T22:00:11.632 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.50895.log 2024-04-23T22:00:11.635 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60849.log: 84.3% -- replaced with /var/log/ceph/cluster2-client.mirror.60849.log.gz 2024-04-23T22:00:11.636 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65465.log 2024-04-23T22:00:11.640 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.50895.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.36453.log 2024-04-23T22:00:11.643 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.65465.log: 91.9% -- replaced with /var/log/ceph/cluster2-client.mirror.65465.log.gz 2024-04-23T22:00:11.643 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75428.log 2024-04-23T22:00:11.645 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.36453.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66003.log 2024-04-23T22:00:11.645 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75428.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.75428.log.gz 2024-04-23T22:00:11.645 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91330.log 2024-04-23T22:00:11.649 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66003.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.66003.log.gz 2024-04-23T22:00:11.649 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71614.log 2024-04-23T22:00:11.650 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.91330.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.91330.log.gz 2024-04-23T22:00:11.650 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105136.log 2024-04-23T22:00:11.653 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.71614.log: 91.8% -- replaced with /var/log/ceph/cluster1-client.mirror.71614.log.gz 2024-04-23T22:00:11.653 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.107614.log 2024-04-23T22:00:11.654 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105136.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.105136.log.gz 2024-04-23T22:00:11.654 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71773.log 2024-04-23T22:00:11.657 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.107614.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.107614.log.gz 2024-04-23T22:00:11.658 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62127.log 2024-04-23T22:00:11.658 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71773.log: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.61723.log.gz 2024-04-23T22:00:11.658 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.71773.log.gz 2024-04-23T22:00:11.662 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.62127.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87547.log 2024-04-23T22:00:11.662 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60061.log 2024-04-23T22:00:11.665 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87547.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.87547.log.gz 2024-04-23T22:00:11.666 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102856.log 2024-04-23T22:00:11.667 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.60061.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55570.log 2024-04-23T22:00:11.667 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102856.log: 86.8% -- replaced with /var/log/ceph/cluster1-client.mirror.102856.log.gz 2024-04-23T22:00:11.670 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63383.log 2024-04-23T22:00:11.671 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.55570.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102158.log 2024-04-23T22:00:11.678 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.63383.log: 84.5% -- replaced with /var/log/ceph/cluster2-client.mirror.36453.log.gz 2024-04-23T22:00:11.681 INFO:teuthology.orchestra.run.smithi150.stderr: 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.58199.log.gz 2024-04-23T22:00:11.682 INFO:teuthology.orchestra.run.smithi150.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.52401.log.gz 2024-04-23T22:00:11.686 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45164.log 2024-04-23T22:00:11.687 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102158.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.102158.log.gz 2024-04-23T22:00:11.690 INFO:teuthology.orchestra.run.smithi150.stderr: 94.8% -- replaced with /var/log/ceph/cluster1-client.mirror.60061.log.gz 2024-04-23T22:00:11.695 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104857.log 2024-04-23T22:00:11.716 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.45164.log: 93.0% -- replaced with /var/log/ceph/cluster1-client.mirror.57576.log.gz 2024-04-23T22:00:11.724 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91770.log 2024-04-23T22:00:11.724 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104857.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.51906.log.gz 2024-04-23T22:00:11.728 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/cluster1-client.mirror.55570.log.gz 2024-04-23T22:00:11.729 INFO:teuthology.orchestra.run.smithi150.stderr: 96.6% -- replaced with /var/log/ceph/cluster2-client.mirror.49527.log.gz 2024-04-23T22:00:11.736 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.62487.log.gz 2024-04-23T22:00:11.738 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.62274.log.gz 2024-04-23T22:00:11.741 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.44785.log 2024-04-23T22:00:11.745 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.91770.log: 92.8% 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.45164.log.gz 2024-04-23T22:00:11.747 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.73127.log.gz 2024-04-23T22:00:11.749 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68308.log 2024-04-23T22:00:11.749 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.57341.log.gz 2024-04-23T22:00:11.749 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.44785.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106061.log 2024-04-23T22:00:11.750 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68308.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.68308.log.gz 2024-04-23T22:00:11.750 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.107004.log 2024-04-23T22:00:11.757 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106061.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.106061.log.gz 2024-04-23T22:00:11.757 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51486.log 2024-04-23T22:00:11.758 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.107004.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.107004.log.gz 2024-04-23T22:00:11.758 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.91770.log.gz 2024-04-23T22:00:11.758 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101348.log 2024-04-23T22:00:11.758 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50297.log 2024-04-23T22:00:11.758 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101348.log: /var/log/ceph/cluster2-client.mirror.51486.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101348.log.gz 2024-04-23T22:00:11.758 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38832.log 2024-04-23T22:00:11.758 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50297.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38222.log 2024-04-23T22:00:11.758 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose --/var/log/ceph/cluster2-client.mirror.38832.log: /var/log/ceph/cluster2-client.mirror.58669.log 2024-04-23T22:00:11.758 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.39034.log 2024-04-23T22:00:11.758 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58669.log: /var/log/ceph/cluster2-client.mirror.38222.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101821.log 2024-04-23T22:00:11.758 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.39034.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98818.log 2024-04-23T22:00:11.758 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101821.log: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.101821.log.gz 2024-04-23T22:00:11.758 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.103785.log 2024-04-23T22:00:11.759 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98818.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.98818.log.gz 2024-04-23T22:00:11.761 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103785.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.103785.log.gz 2024-04-23T22:00:11.761 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65951.log 2024-04-23T22:00:11.761 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46383.log 2024-04-23T22:00:11.769 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.65951.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104495.log 2024-04-23T22:00:11.788 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.46383.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68420.log 2024-04-23T22:00:11.788 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104495.log: 83.3% -- replaced with /var/log/ceph/cluster2-client.mirror.104495.log.gz 2024-04-23T22:00:11.788 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105718.log 2024-04-23T22:00:11.792 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68420.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.68420.log.gz 2024-04-23T22:00:11.793 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45005.log 2024-04-23T22:00:11.795 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% 92.8% -- replaced with /var/log/ceph/cluster2-client.mirror.44549.log.gz 2024-04-23T22:00:11.797 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.50895.log.gz 2024-04-23T22:00:11.798 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105718.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.105718.log.gz 2024-04-23T22:00:11.799 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35951.log 2024-04-23T22:00:11.799 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.45005.log: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.63644.log.gz 2024-04-23T22:00:11.801 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82002.log 2024-04-23T22:00:11.801 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.35951.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75025.log 2024-04-23T22:00:11.802 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85612.log 2024-04-23T22:00:11.802 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75025.log: /var/log/ceph/cluster2-client.mirror.82002.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.82002.log.gz 2024-04-23T22:00:11.803 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81638.log 2024-04-23T22:00:11.805 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.85612.log: 91.5% -- replaced with /var/log/ceph/cluster1-client.mirror.85612.log.gz 2024-04-23T22:00:11.805 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.37142.log 2024-04-23T22:00:11.815 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81638.log: 88.1% -- replaced with /var/log/ceph/cluster1-client.admin.44785.log.gz 2024-04-23T22:00:11.815 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.38222.log.gz 2024-04-23T22:00:11.817 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.38832.log.gz 2024-04-23T22:00:11.818 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55188.log 2024-04-23T22:00:11.818 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.39034.log.gz 2024-04-23T22:00:11.819 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.37142.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.45005.log.gz 2024-04-23T22:00:11.821 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster1-client.admin.46383.log.gz 2024-04-23T22:00:11.821 INFO:teuthology.orchestra.run.smithi150.stderr: 80.0% -- replaced with /var/log/ceph/cluster1-client.admin.35951.log.gz 2024-04-23T22:00:11.834 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69687.log 2024-04-23T22:00:11.843 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55188.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92576.log 2024-04-23T22:00:11.856 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69687.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.87894.log 2024-04-23T22:00:11.856 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.69687.log.gz 2024-04-23T22:00:11.857 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.92576.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.92576.log.gz 2024-04-23T22:00:11.867 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106294.log 2024-04-23T22:00:11.871 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.87894.log: 92.3% 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.75025.log.gz 2024-04-23T22:00:11.871 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93579.log 2024-04-23T22:00:11.871 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.87894.log.gz 2024-04-23T22:00:11.871 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.106294.log: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.106294.log.gz 2024-04-23T22:00:11.871 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56059.log 2024-04-23T22:00:11.895 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.93579.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58481.log 2024-04-23T22:00:11.904 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56059.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42366.log 2024-04-23T22:00:11.917 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58481.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58355.log 2024-04-23T22:00:11.923 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.42366.log: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.55188.log.gz 2024-04-23T22:00:11.928 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88529.log 2024-04-23T22:00:11.936 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58355.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56578.log 2024-04-23T22:00:11.944 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88529.log: 87.9%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48766.log 2024-04-23T22:00:11.949 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56578.log: -- replaced with /var/log/ceph/cluster2-client.mirror.88529.log.gz 2024-04-23T22:00:11.953 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89398.log 2024-04-23T22:00:11.964 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48766.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61642.log 2024-04-23T22:00:11.965 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89398.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.89398.log.gz 2024-04-23T22:00:11.977 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56362.log 2024-04-23T22:00:11.978 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.61642.log: 84.5% -- replaced with /var/log/ceph/cluster2-client.mirror.61642.log.gz 2024-04-23T22:00:11.978 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.56059.log.gz 2024-04-23T22:00:11.979 INFO:teuthology.orchestra.run.smithi150.stderr: 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.58355.log.gz 2024-04-23T22:00:11.980 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.58481.log.gz 2024-04-23T22:00:11.981 INFO:teuthology.orchestra.run.smithi150.stderr: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.56578.log.gz 2024-04-23T22:00:11.981 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.48766.log.gz 2024-04-23T22:00:11.983 INFO:teuthology.orchestra.run.smithi150.stderr: 92.9% -- replaced with /var/log/ceph/cluster1-client.mirror.49462.log.gz 2024-04-23T22:00:11.986 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97939.log 2024-04-23T22:00:12.002 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56362.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97424.log 2024-04-23T22:00:12.003 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97939.log: 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.97939.log.gz 2024-04-23T22:00:12.012 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45558.log 2024-04-23T22:00:12.020 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97424.log: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.97424.log.gz 2024-04-23T22:00:12.022 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.93579.log.gz 2024-04-23T22:00:12.023 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104769.log 2024-04-23T22:00:12.023 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.65951.log.gz 2024-04-23T22:00:12.025 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.45558.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59218.log 2024-04-23T22:00:12.026 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104769.log: 91.4% -- replaced with /var/log/ceph/cluster2-client.mirror.104769.log.gz 2024-04-23T22:00:12.027 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50332.log 2024-04-23T22:00:12.029 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.59218.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71294.log 2024-04-23T22:00:12.029 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50332.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67665.log 2024-04-23T22:00:12.029 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71294.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.80795.log 2024-04-23T22:00:12.029 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.67665.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.67665.log.gz 2024-04-23T22:00:12.029 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.71294.log.gz 2024-04-23T22:00:12.032 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101326.log 2024-04-23T22:00:12.033 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38424.log 2024-04-23T22:00:12.033 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101326.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.101326.log.gz 2024-04-23T22:00:12.036 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.80795.log: gzip -5 --verbose -- 91.9% /var/log/ceph/cluster1-client.mirror.75707.log 2024-04-23T22:00:12.036 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.80795.log.gz 2024-04-23T22:00:12.036 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.38424.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52807.log 2024-04-23T22:00:12.036 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.75707.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102608.log 2024-04-23T22:00:12.038 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52807.log: 91.9% -- replaced with /var/log/ceph/cluster1-client.mirror.75707.log.gz 2024-04-23T22:00:12.038 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70188.log 2024-04-23T22:00:12.038 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102608.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102608.log.gz 2024-04-23T22:00:12.040 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82760.log 2024-04-23T22:00:12.040 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.70188.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.70188.log.gz 2024-04-23T22:00:12.042 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57169.log 2024-04-23T22:00:12.046 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.82760.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44028.log 2024-04-23T22:00:12.046 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57169.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.82760.log.gz 2024-04-23T22:00:12.046 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100754.log 2024-04-23T22:00:12.046 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.93512.log 2024-04-23T22:00:12.046 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100754.log: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.100754.log.gz 2024-04-23T22:00:12.046 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44028.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96484.log 2024-04-23T22:00:12.048 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.93512.log: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.93512.log.gz 2024-04-23T22:00:12.048 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104514.log 2024-04-23T22:00:12.048 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.96484.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59303.log 2024-04-23T22:00:12.048 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104514.log: 88.1% 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.104514.log.gz 2024-04-23T22:00:12.049 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.96484.log.gz 2024-04-23T22:00:12.050 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68104.log 2024-04-23T22:00:12.050 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.59303.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74587.log 2024-04-23T22:00:12.058 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.68104.log: 87.9% -- replaced with /var/log/ceph/cluster1-client.mirror.68104.log.gzgzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101416.log 2024-04-23T22:00:12.058 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.058 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74587.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93696.log 2024-04-23T22:00:12.058 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101416.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.74587.log.gz 2024-04-23T22:00:12.058 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101416.log.gzgzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65929.log 2024-04-23T22:00:12.058 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.058 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.93696.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.86189.log 2024-04-23T22:00:12.058 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.65929.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.93696.log.gz 2024-04-23T22:00:12.058 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84305.log 2024-04-23T22:00:12.058 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.86189.log: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.86189.log.gz 2024-04-23T22:00:12.058 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41070.log 2024-04-23T22:00:12.058 INFO:teuthology.orchestra.run.smithi150.stderr: 91.8%/var/log/ceph/cluster2-client.mirror.84305.log: -- replaced with /var/log/ceph/cluster1-client.mirror.65929.log.gz 2024-04-23T22:00:12.058 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.84305.log.gzgzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106833.log 2024-04-23T22:00:12.058 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.059 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41070.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85110.log 2024-04-23T22:00:12.059 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.106833.log: gzip -5 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106833.log.gz 2024-04-23T22:00:12.059 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.36002.log 2024-04-23T22:00:12.059 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85110.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57056.log 2024-04-23T22:00:12.059 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.85110.log.gz 2024-04-23T22:00:12.059 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.36002.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76249.log 2024-04-23T22:00:12.059 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57056.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103648.log 2024-04-23T22:00:12.059 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76249.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57943.log 2024-04-23T22:00:12.059 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.76249.log.gz 2024-04-23T22:00:12.067 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.103648.log: 83.3% -- replaced with /var/log/ceph/cluster2-client.mirror.103648.log.gz 2024-04-23T22:00:12.067 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54768.log 2024-04-23T22:00:12.067 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57943.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58731.log 2024-04-23T22:00:12.067 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.54768.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98918.log 2024-04-23T22:00:12.067 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58731.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88784.log 2024-04-23T22:00:12.067 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98918.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51544.log 2024-04-23T22:00:12.067 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88784.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.88784.log.gz 2024-04-23T22:00:12.067 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.42366.log.gz 2024-04-23T22:00:12.067 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.98918.log.gz 2024-04-23T22:00:12.067 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98675.log 2024-04-23T22:00:12.067 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.44028.log.gz 2024-04-23T22:00:12.067 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.51544.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.85567.log 2024-04-23T22:00:12.067 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98675.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104809.log 2024-04-23T22:00:12.068 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.98675.log.gz 2024-04-23T22:00:12.068 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.85567.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70242.log 2024-04-23T22:00:12.068 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104809.log: 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.85567.log.gz 87.9% 2024-04-23T22:00:12.068 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.104809.log.gz 2024-04-23T22:00:12.068 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58523.log 2024-04-23T22:00:12.068 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103271.log 2024-04-23T22:00:12.068 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58523.log: /var/log/ceph/cluster2-client.mirror.70242.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.38424.log.gz 2024-04-23T22:00:12.068 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.70242.log.gz 2024-04-23T22:00:12.068 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91665.log 2024-04-23T22:00:12.070 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.103271.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.103271.log.gz 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.54768.log.gz 2024-04-23T22:00:12.070 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.070 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63047.log 2024-04-23T22:00:12.071 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.91665.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.91665.log.gz 2024-04-23T22:00:12.071 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70847.log 2024-04-23T22:00:12.071 INFO:teuthology.orchestra.run.smithi150.stderr: 84.5% -- replaced with /var/log/ceph/cluster2-client.mirror.59218.log.gz 2024-04-23T22:00:12.071 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.63047.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.57056.log.gz 2024-04-23T22:00:12.074 INFO:teuthology.orchestra.run.smithi150.stderr: 94.8% -- replaced with /var/log/ceph/cluster1-client.mirror.63047.log.gz 2024-04-23T22:00:12.074 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105821.log 2024-04-23T22:00:12.074 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97359.log 2024-04-23T22:00:12.074 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105821.log: /var/log/ceph/cluster2-client.mirror.70847.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.105821.log.gz 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.70847.log.gz 2024-04-23T22:00:12.074 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.074 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85912.log 2024-04-23T22:00:12.075 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97359.log: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.97359.log.gz 2024-04-23T22:00:12.076 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.37712.log 2024-04-23T22:00:12.077 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85912.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.85912.log.gz 2024-04-23T22:00:12.077 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61165.log 2024-04-23T22:00:12.080 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.37712.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49057.log 2024-04-23T22:00:12.080 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77381.log 2024-04-23T22:00:12.080 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.49057.log: /var/log/ceph/cluster1-client.mirror.61165.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100399.log 2024-04-23T22:00:12.080 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77381.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.77381.log.gz 2024-04-23T22:00:12.080 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96434.log 2024-04-23T22:00:12.080 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100399.log: 94.5% -- replaced with /var/log/ceph/cluster1-client.mirror.61165.log.gz 2024-04-23T22:00:12.108 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77327.log 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.100399.log.gz 2024-04-23T22:00:12.108 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.108 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48828.log 2024-04-23T22:00:12.108 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77327.log: /var/log/ceph/cluster2-client.mirror.96434.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.77327.log.gz 2024-04-23T22:00:12.108 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.96434.log.gz 2024-04-23T22:00:12.108 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95734.log 2024-04-23T22:00:12.108 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48828.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63816.log 2024-04-23T22:00:12.108 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46414.log 2024-04-23T22:00:12.108 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.63816.log: /var/log/ceph/cluster2-client.mirror.95734.log: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.81638.log.gz 84.3% -- replaced with /var/log/ceph/cluster2-client.mirror.63816.log.gz 2024-04-23T22:00:12.108 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.108 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.95734.log.gz 2024-04-23T22:00:12.109 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58313.log 2024-04-23T22:00:12.109 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.46414.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38797.log 2024-04-23T22:00:12.109 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster2-client.mirror.58313.log: -- /var/log/ceph/cluster2-client.mirror.94388.log 2024-04-23T22:00:12.109 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88199.log 2024-04-23T22:00:12.109 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94388.log: /var/log/ceph/cluster2-client.mirror.38797.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81788.log 2024-04-23T22:00:12.109 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.94388.log.gz 2024-04-23T22:00:12.109 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88199.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72539.log 2024-04-23T22:00:12.109 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.88199.log.gz 2024-04-23T22:00:12.109 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98413.log 2024-04-23T22:00:12.109 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81788.log: /var/log/ceph/cluster2-client.mirror.72539.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98747.log 2024-04-23T22:00:12.109 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98413.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.72539.log.gz 2024-04-23T22:00:12.109 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.81788.log.gzgzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101888.log 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.98413.log.gz 2024-04-23T22:00:12.109 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.109 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.109 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98747.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60255.log 2024-04-23T22:00:12.110 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101888.log: gzip -5 --verbose -- 86.8% /var/log/ceph/cluster2-client.mirror.53858.log 2024-04-23T22:00:12.110 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.101888.log.gz 2024-04-23T22:00:12.110 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93866.log 2024-04-23T22:00:12.110 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60255.log: /var/log/ceph/cluster2-client.mirror.53858.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61499.log 2024-04-23T22:00:12.110 INFO:teuthology.orchestra.run.smithi150.stderr: 90.6% -- replaced with /var/log/ceph/cluster2-client.mirror.98747.log.gz 2024-04-23T22:00:12.110 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.93866.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99711.log 2024-04-23T22:00:12.110 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.93866.log.gz 2024-04-23T22:00:12.110 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.61499.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91022.log 2024-04-23T22:00:12.110 INFO:teuthology.orchestra.run.smithi150.stderr: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.61499.log.gz 2024-04-23T22:00:12.110 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48634.log 2024-04-23T22:00:12.110 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99711.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57748.log 2024-04-23T22:00:12.110 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34392.log 2024-04-23T22:00:12.110 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.99711.log.gz 2024-04-23T22:00:12.110 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48634.log: /var/log/ceph/cluster1-client.mirror.57748.log: /var/log/ceph/cluster2-client.mirror.91022.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65104.log 2024-04-23T22:00:12.110 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.91022.log.gz/var/log/ceph/cluster2-client.admin.34392.log: 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34392.log.gz 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86447.log 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100583.log 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.65104.log: /var/log/ceph/cluster2-client.mirror.86447.log: 1.3% -- replaced with /var/log/ceph/cluster1-client.mirror.65104.log.gz 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104031.log 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9%/var/log/ceph/cluster2-client.mirror.100583.log: -- replaced with /var/log/ceph/cluster2-client.mirror.86447.log.gz 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96519.log 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80602.log 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.96519.log: /var/log/ceph/cluster2-client.mirror.104031.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.96519.log.gz 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66826.log 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.80602.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.80602.log.gz 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42864.log 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66826.log: 91.5% -- replaced with /var/log/ceph/cluster2-client.mirror.104031.log.gz 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.66826.log.gz 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68803.log 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80029.log 2024-04-23T22:00:12.111 INFO:teuthology.orchestra.run.smithi150.stderr: 90.6% -- replaced with /var/log/ceph/cluster2-client.mirror.100583.log.gz/var/log/ceph/cluster2-client.mirror.42864.log: 2024-04-23T22:00:12.112 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.68803.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52358.log 2024-04-23T22:00:12.112 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49304.log 2024-04-23T22:00:12.112 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.52358.log: /var/log/ceph/cluster2-client.mirror.80029.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41264.log 2024-04-23T22:00:12.112 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.80029.log.gz 2024-04-23T22:00:12.112 INFO:teuthology.orchestra.run.smithi150.stderr: 91.9% -- replaced with /var/log/ceph/cluster1-client.mirror.68803.log.gz 2024-04-23T22:00:12.112 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.49304.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.107572.log 2024-04-23T22:00:12.112 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41264.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77700.log 2024-04-23T22:00:12.112 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster2-client.admin.107572.log: -- /var/log/ceph/cluster2-client.mirror.43605.log 2024-04-23T22:00:12.112 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.107572.log.gz 2024-04-23T22:00:12.112 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101011.log 2024-04-23T22:00:12.112 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77700.log: /var/log/ceph/cluster2-client.mirror.43605.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60485.log 2024-04-23T22:00:12.112 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101011.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98555.log 2024-04-23T22:00:12.112 INFO:teuthology.orchestra.run.smithi150.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.101011.log.gz 87.9%/var/log/ceph/cluster2-client.mirror.60485.log: -- replaced with /var/log/ceph/cluster2-client.mirror.77700.log.gz 2024-04-23T22:00:12.112 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.112 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49223.log 2024-04-23T22:00:12.113 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98555.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49442.log 2024-04-23T22:00:12.113 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.49223.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.98555.log.gz 2024-04-23T22:00:12.113 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49691.log 2024-04-23T22:00:12.113 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92361.log 2024-04-23T22:00:12.113 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.49442.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94077.log 2024-04-23T22:00:12.113 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.92361.log: /var/log/ceph/cluster1-client.mirror.49691.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.92361.log.gz 2024-04-23T22:00:12.113 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45450.log 2024-04-23T22:00:12.113 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster2-client.mirror.94077.log: -- /var/log/ceph/cluster2-client.mirror.90185.log 2024-04-23T22:00:12.113 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85245.log 2024-04-23T22:00:12.113 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.45450.log: /var/log/ceph/cluster2-client.mirror.90185.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.90185.log.gz 2024-04-23T22:00:12.113 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87597.log 2024-04-23T22:00:12.113 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88869.log 2024-04-23T22:00:12.113 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87597.log: /var/log/ceph/cluster2-client.mirror.85245.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.87597.log.gz 2024-04-23T22:00:12.113 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.85245.log.gz 2024-04-23T22:00:12.113 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80954.log 2024-04-23T22:00:12.115 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88869.log: gzip -5 --verbose -- 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.88869.log.gz 2024-04-23T22:00:12.115 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster1-client.mirror.69958.log 2024-04-23T22:00:12.115 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44614.log 2024-04-23T22:00:12.117 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.69958.log: /var/log/ceph/cluster2-client.mirror.80954.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.80954.log.gz 2024-04-23T22:00:12.117 INFO:teuthology.orchestra.run.smithi150.stderr: 91.8% -- replaced with /var/log/ceph/cluster1-client.mirror.69958.log.gz 2024-04-23T22:00:12.117 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.89922.log 2024-04-23T22:00:12.118 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44614.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65408.log 2024-04-23T22:00:12.118 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105381.log 2024-04-23T22:00:12.124 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.89922.log: /var/log/ceph/cluster2-client.mirror.65408.log: 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.89922.log.gz 2024-04-23T22:00:12.124 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.65408.log.gz 2024-04-23T22:00:12.124 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50623.log 2024-04-23T22:00:12.124 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.105381.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.37576.log 2024-04-23T22:00:12.124 INFO:teuthology.orchestra.run.smithi150.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.105381.log.gz 2024-04-23T22:00:12.124 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50623.log: /var/log/ceph/cluster2-client.mirror.37576.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92541.log 2024-04-23T22:00:12.124 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59009.log 2024-04-23T22:00:12.124 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.92541.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41299.log 2024-04-23T22:00:12.124 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.92541.log.gz 2024-04-23T22:00:12.124 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.59009.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44591.log 2024-04-23T22:00:12.124 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41299.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90575.log 2024-04-23T22:00:12.124 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44591.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55765.log 2024-04-23T22:00:12.124 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90575.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79624.log 2024-04-23T22:00:12.125 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.90575.log.gz 2024-04-23T22:00:12.126 INFO:teuthology.orchestra.run.smithi150.stderr: 95.9% -- replaced with /var/log/ceph/cluster2-client.mirror.63383.log.gz 2024-04-23T22:00:12.126 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.55765.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58225.log 2024-04-23T22:00:12.137 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.79624.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.107656.log 2024-04-23T22:00:12.138 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58225.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.79624.log.gz 2024-04-23T22:00:12.138 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105326.log 2024-04-23T22:00:12.138 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67239.log 2024-04-23T22:00:12.138 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.107656.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.107656.log.gz 2024-04-23T22:00:12.138 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50235.log 2024-04-23T22:00:12.138 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.67239.log: /var/log/ceph/cluster1-client.mirror.105326.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.39287.log 2024-04-23T22:00:12.138 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50235.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.105326.log.gzgzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41555.log 2024-04-23T22:00:12.138 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.138 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54348.log 2024-04-23T22:00:12.138 INFO:teuthology.orchestra.run.smithi150.stderr: 91.9% -- replaced with /var/log/ceph/cluster1-client.mirror.67239.log.gz 2024-04-23T22:00:12.138 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41555.log: /var/log/ceph/cluster1-client.admin.39287.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81504.log 2024-04-23T22:00:12.138 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75104.log 2024-04-23T22:00:12.138 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81504.log: /var/log/ceph/cluster2-client.mirror.54348.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40391.log 2024-04-23T22:00:12.139 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.81504.log.gz/var/log/ceph/cluster2-client.mirror.75104.log: 2024-04-23T22:00:12.139 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97910.log 2024-04-23T22:00:12.139 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.75104.log.gz 2024-04-23T22:00:12.139 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53928.log 2024-04-23T22:00:12.139 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40391.log: /var/log/ceph/cluster2-client.mirror.97910.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94966.log 2024-04-23T22:00:12.139 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.53928.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103990.log 2024-04-23T22:00:12.139 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91597.log 2024-04-23T22:00:12.139 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.103990.log: /var/log/ceph/cluster2-client.mirror.94966.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.103990.log.gz 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.94966.log.gz 2024-04-23T22:00:12.139 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.139 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99782.log 2024-04-23T22:00:12.139 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.91597.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.97910.log.gz 2024-04-23T22:00:12.139 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89668.log 2024-04-23T22:00:12.139 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99782.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35985.log 2024-04-23T22:00:12.139 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89668.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.99782.log.gz 2024-04-23T22:00:12.139 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.89668.log.gz 2024-04-23T22:00:12.143 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106169.log 2024-04-23T22:00:12.143 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.35985.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102068.log 2024-04-23T22:00:12.143 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster2-client.mirror.106169.log: -- /var/log/ceph/cluster2-client.mirror.46632.log 2024-04-23T22:00:12.143 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102068.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102068.log.gz 2024-04-23T22:00:12.143 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106359.log 2024-04-23T22:00:12.143 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46632.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.103081.log 2024-04-23T22:00:12.144 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106359.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.106359.log.gz 2024-04-23T22:00:12.145 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84637.log 2024-04-23T22:00:12.146 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103081.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.103081.log.gz 2024-04-23T22:00:12.147 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50873.log 2024-04-23T22:00:12.147 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.84637.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.84637.log.gz 2024-04-23T22:00:12.148 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73060.log 2024-04-23T22:00:12.149 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50873.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93126.log 2024-04-23T22:00:12.151 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.73060.log: 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.73060.log.gz 2024-04-23T22:00:12.151 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99944.log 2024-04-23T22:00:12.152 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.93126.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.93126.log.gz 2024-04-23T22:00:12.153 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105062.log 2024-04-23T22:00:12.157 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99944.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.99944.log.gz 2024-04-23T22:00:12.158 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99995.log 2024-04-23T22:00:12.163 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105062.log: 91.3% -- replaced with /var/log/ceph/cluster2-client.mirror.105062.log.gz 2024-04-23T22:00:12.163 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94342.log 2024-04-23T22:00:12.164 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99995.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.99995.log.gz 2024-04-23T22:00:12.164 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88021.log 2024-04-23T22:00:12.165 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94342.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.94342.log.gz 2024-04-23T22:00:12.166 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.39941.log 2024-04-23T22:00:12.166 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88021.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.88021.log.gz 2024-04-23T22:00:12.169 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56425.log 2024-04-23T22:00:12.172 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.39941.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87916.log 2024-04-23T22:00:12.174 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56425.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.78223.log 2024-04-23T22:00:12.185 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87916.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98082.log 2024-04-23T22:00:12.185 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.78223.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.78223.log.gz 2024-04-23T22:00:12.187 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73790.log 2024-04-23T22:00:12.188 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98082.log: 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.98082.log.gz 2024-04-23T22:00:12.188 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47214.log 2024-04-23T22:00:12.194 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73790.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.73790.log.gz 2024-04-23T22:00:12.194 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35927.log 2024-04-23T22:00:12.194 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.47214.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102991.log 2024-04-23T22:00:12.194 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.35927.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35927.log.gz 2024-04-23T22:00:12.194 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55398.log 2024-04-23T22:00:12.194 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102991.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102991.log.gz 2024-04-23T22:00:12.194 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34941.log 2024-04-23T22:00:12.194 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55398.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.92113.log 2024-04-23T22:00:12.194 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34941.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34941.log.gz 2024-04-23T22:00:12.194 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.94077.log.gz 2024-04-23T22:00:12.195 INFO:teuthology.orchestra.run.smithi150.stderr: -- /var/log/ceph/cluster2-client.mirror.66435.log 2024-04-23T22:00:12.195 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.92113.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.92113.log.gz 2024-04-23T22:00:12.195 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105239.log 2024-04-23T22:00:12.195 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66435.log: 92.0% -- replaced with /var/log/ceph/cluster2-client.mirror.66435.log.gz 2024-04-23T22:00:12.203 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89718.log 2024-04-23T22:00:12.205 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105239.log: 88.0% 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.42864.log.gz 2024-04-23T22:00:12.206 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.58313.log.gz 2024-04-23T22:00:12.206 INFO:teuthology.orchestra.run.smithi150.stderr: 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.58523.log.gz 2024-04-23T22:00:12.207 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.37712.log.gz 2024-04-23T22:00:12.209 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster1-client.admin.46414.log.gz 2024-04-23T22:00:12.210 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.38797.log.gz 2024-04-23T22:00:12.212 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.46632.log.gz 2024-04-23T22:00:12.213 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.48634.log.gz 2024-04-23T22:00:12.213 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.48828.log.gz 2024-04-23T22:00:12.214 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster1-client.admin.39287.log.gz 2024-04-23T22:00:12.215 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52877.log 2024-04-23T22:00:12.215 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.49057.log.gz 2024-04-23T22:00:12.215 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89718.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.89718.log.gz 2024-04-23T22:00:12.216 INFO:teuthology.orchestra.run.smithi150.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.105239.log.gz 2024-04-23T22:00:12.218 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75825.log 2024-04-23T22:00:12.218 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52877.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54593.log 2024-04-23T22:00:12.218 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75825.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.75825.log.gz 2024-04-23T22:00:12.220 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35025.log 2024-04-23T22:00:12.220 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.54593.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99831.log 2024-04-23T22:00:12.220 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.35025.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35025.log.gz 2024-04-23T22:00:12.220 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41943.log 2024-04-23T22:00:12.224 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99831.log: 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.99831.log.gz 2024-04-23T22:00:12.225 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104093.log 2024-04-23T22:00:12.225 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41943.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70621.log 2024-04-23T22:00:12.226 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104093.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.104093.log.gz 2024-04-23T22:00:12.226 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.62127.log.gz 2024-04-23T22:00:12.227 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.103103.log 2024-04-23T22:00:12.228 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.70621.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.70621.log.gz 2024-04-23T22:00:12.229 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75003.log 2024-04-23T22:00:12.231 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103103.log: /var/log/ceph/cluster1-client.mirror.75003.log: 91.6% -- replaced with /var/log/ceph/cluster1-client.mirror.75003.log.gz 2024-04-23T22:00:12.231 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.103103.log.gz 2024-04-23T22:00:12.232 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45535.log 2024-04-23T22:00:12.234 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67870.log 2024-04-23T22:00:12.234 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.45535.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98533.log 2024-04-23T22:00:12.234 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62646.log 2024-04-23T22:00:12.235 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98533.log: /var/log/ceph/cluster2-client.mirror.67870.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.67870.log.gz 2024-04-23T22:00:12.235 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.98533.log.gz 2024-04-23T22:00:12.236 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.104857.log.gz 2024-04-23T22:00:12.242 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54453.log 2024-04-23T22:00:12.242 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.62646.log: gzip -5 --verbose -- 84.4% /var/log/ceph/cluster1-client.admin.50926.log 2024-04-23T22:00:12.242 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.62646.log.gz 2024-04-23T22:00:12.242 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.35319.log 2024-04-23T22:00:12.242 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster1-client.admin.50926.log: -- /var/log/ceph/cluster2-client.mirror.95043.log 2024-04-23T22:00:12.243 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83997.log 2024-04-23T22:00:12.243 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95043.log: /var/log/ceph/cluster2-client.admin.35319.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.35319.log.gz 2024-04-23T22:00:12.243 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.95043.log.gz 2024-04-23T22:00:12.243 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105619.log 2024-04-23T22:00:12.243 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38933.log 2024-04-23T22:00:12.243 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.105619.log: /var/log/ceph/cluster2-client.mirror.83997.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.105619.log.gz 2024-04-23T22:00:12.243 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.54453.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89262.log 2024-04-23T22:00:12.243 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.38933.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72004.log 2024-04-23T22:00:12.243 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89262.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50006.log 2024-04-23T22:00:12.255 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.72004.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.72004.log.gz 2024-04-23T22:00:12.255 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.100922.log 2024-04-23T22:00:12.255 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50006.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.49505.log 2024-04-23T22:00:12.255 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.100922.log: 1.3% -- replaced with /var/log/ceph/cluster1-client.mirror.100922.log.gz 2024-04-23T22:00:12.255 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70882.log 2024-04-23T22:00:12.255 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.49505.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.89262.log.gzgzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102226.log 2024-04-23T22:00:12.255 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.256 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.70882.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40232.log 2024-04-23T22:00:12.256 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.70882.log.gz 2024-04-23T22:00:12.256 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102226.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102226.log.gz 2024-04-23T22:00:12.256 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76470.log 2024-04-23T22:00:12.256 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40232.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102946.log 2024-04-23T22:00:12.256 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.76470.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50429.log 2024-04-23T22:00:12.256 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102946.log: 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.76470.log.gz 2024-04-23T22:00:12.256 INFO:teuthology.orchestra.run.smithi150.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.102946.log.gz 2024-04-23T22:00:12.256 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95571.log 2024-04-23T22:00:12.256 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50429.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77937.log 2024-04-23T22:00:12.256 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95571.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72725.log 2024-04-23T22:00:12.256 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77937.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.95571.log.gz 2024-04-23T22:00:12.256 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67902.log 2024-04-23T22:00:12.256 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.72725.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.72725.log.gz 2024-04-23T22:00:12.256 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34143.log 2024-04-23T22:00:12.257 INFO:teuthology.orchestra.run.smithi150.stderr: 91.8% -- replaced with /var/log/ceph/cluster2-client.mirror.77937.log.gz 2024-04-23T22:00:12.257 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.67902.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98226.log 2024-04-23T22:00:12.257 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.67902.log.gz 2024-04-23T22:00:12.257 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34143.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73825.log 2024-04-23T22:00:12.257 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98226.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34143.log.gz 2024-04-23T22:00:12.257 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45907.log 2024-04-23T22:00:12.257 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73825.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.73825.log.gz 2024-04-23T22:00:12.257 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81066.log 2024-04-23T22:00:12.276 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.45907.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49385.log 2024-04-23T22:00:12.276 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81066.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.81066.log.gz 2024-04-23T22:00:12.276 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61700.log 2024-04-23T22:00:12.276 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.49385.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105010.log 2024-04-23T22:00:12.276 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.61700.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.98226.log.gz 2024-04-23T22:00:12.276 INFO:teuthology.orchestra.run.smithi150.stderr: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.61700.log.gz 2024-04-23T22:00:12.277 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55433.log 2024-04-23T22:00:12.277 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.105010.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50785.log 2024-04-23T22:00:12.277 INFO:teuthology.orchestra.run.smithi150.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.105010.log.gz 2024-04-23T22:00:12.277 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55433.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42926.log 2024-04-23T22:00:12.277 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.50785.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77880.log 2024-04-23T22:00:12.277 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100086.log 2024-04-23T22:00:12.277 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77880.log: /var/log/ceph/cluster2-client.mirror.42926.log: gzip -5 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.77880.log.gz 2024-04-23T22:00:12.277 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/cluster2-client.mirror.58790.log 2024-04-23T22:00:12.277 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100086.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81185.log 2024-04-23T22:00:12.277 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58790.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73028.log 2024-04-23T22:00:12.277 INFO:teuthology.orchestra.run.smithi150.stderr: 90.6% -- replaced with /var/log/ceph/cluster2-client.mirror.100086.log.gz 2024-04-23T22:00:12.277 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106212.log 2024-04-23T22:00:12.277 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73028.log: /var/log/ceph/cluster2-client.mirror.81185.log: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.56362.log.gz 2024-04-23T22:00:12.277 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.73028.log.gz 2024-04-23T22:00:12.277 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.81185.log.gz 2024-04-23T22:00:12.278 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71927.log 2024-04-23T22:00:12.278 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74891.log 2024-04-23T22:00:12.278 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71927.log: /var/log/ceph/cluster1-client.mirror.106212.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.71927.log.gz 2024-04-23T22:00:12.278 INFO:teuthology.orchestra.run.smithi150.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.106212.log.gz 2024-04-23T22:00:12.278 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40620.log 2024-04-23T22:00:12.278 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85450.log 2024-04-23T22:00:12.278 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40620.log: /var/log/ceph/cluster2-client.mirror.74891.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88291.log 2024-04-23T22:00:12.278 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85450.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.74891.log.gz 2024-04-23T22:00:12.278 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.85450.log.gz 2024-04-23T22:00:12.278 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40003.log 2024-04-23T22:00:12.278 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83975.log 2024-04-23T22:00:12.278 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40003.log: /var/log/ceph/cluster2-client.mirror.88291.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97310.log 2024-04-23T22:00:12.278 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.83975.log: 92.1% -- replaced with /var/log/ceph/cluster2-client.mirror.88291.log.gz 2024-04-23T22:00:12.278 INFO:teuthology.orchestra.run.smithi150.stderr: 91.7% -- replaced with /var/log/ceph/cluster1-client.mirror.83975.log.gz 2024-04-23T22:00:12.278 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101146.log 2024-04-23T22:00:12.279 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97310.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.97310.log.gz 2024-04-23T22:00:12.279 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52982.log 2024-04-23T22:00:12.279 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101146.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77562.log 2024-04-23T22:00:12.279 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101146.log.gz 2024-04-23T22:00:12.279 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52982.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94552.log 2024-04-23T22:00:12.279 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5/var/log/ceph/cluster2-client.mirror.77562.log: --verbose -- /var/log/ceph/cluster2-client.mirror.80637.log 2024-04-23T22:00:12.279 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94552.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.94552.log.gz 2024-04-23T22:00:12.279 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103855.log 2024-04-23T22:00:12.279 INFO:teuthology.orchestra.run.smithi150.stderr: 91.7% -- replaced with /var/log/ceph/cluster2-client.mirror.77562.log.gz/var/log/ceph/cluster2-client.mirror.80637.log: 2024-04-23T22:00:12.291 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50740.log 2024-04-23T22:00:12.291 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.80637.log.gz/var/log/ceph/cluster2-client.mirror.103855.log: 2024-04-23T22:00:12.291 INFO:teuthology.orchestra.run.smithi150.stderr: 86.1% -- replaced with /var/log/ceph/cluster2-client.mirror.103855.log.gz 2024-04-23T22:00:12.291 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44506.log 2024-04-23T22:00:12.291 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster1-client.mirror.50740.log: -- /var/log/ceph/cluster1-client.mirror.68149.log 2024-04-23T22:00:12.291 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57920.log 2024-04-23T22:00:12.292 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.68149.log: /var/log/ceph/cluster2-client.mirror.44506.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100137.log 2024-04-23T22:00:12.292 INFO:teuthology.orchestra.run.smithi150.stderr: 91.8% -- replaced with /var/log/ceph/cluster1-client.mirror.68149.log.gz 2024-04-23T22:00:12.292 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57920.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58049.log 2024-04-23T22:00:12.292 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100137.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.100137.log.gz 2024-04-23T22:00:12.292 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60790.log 2024-04-23T22:00:12.292 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster1-client.mirror.58049.log: -- /var/log/ceph/cluster2-client.mirror.95536.log 2024-04-23T22:00:12.292 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.34983.log 2024-04-23T22:00:12.292 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95536.log: /var/log/ceph/cluster2-client.mirror.60790.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.95536.log.gz 2024-04-23T22:00:12.292 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76515.log 2024-04-23T22:00:12.292 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.34983.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.34983.log.gz 2024-04-23T22:00:12.292 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73547.log 2024-04-23T22:00:12.292 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.76515.log: 92.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76515.log.gz 2024-04-23T22:00:12.292 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95358.log 2024-04-23T22:00:12.292 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73547.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.73547.log.gz 2024-04-23T22:00:12.293 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.36207.log 2024-04-23T22:00:12.294 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95358.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.95358.log.gz 2024-04-23T22:00:12.294 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72504.log 2024-04-23T22:00:12.299 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.36207.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47796.log 2024-04-23T22:00:12.299 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.72504.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.72504.log.gz 2024-04-23T22:00:12.300 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69757.log 2024-04-23T22:00:12.302 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.47796.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.40391.log.gz 2024-04-23T22:00:12.302 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57513.log 2024-04-23T22:00:12.304 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69757.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.40620.log.gz 2024-04-23T22:00:12.304 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.69757.log.gz 2024-04-23T22:00:12.304 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55793.log 2024-04-23T22:00:12.305 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57513.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.103058.log 2024-04-23T22:00:12.308 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55793.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34519.log 2024-04-23T22:00:12.309 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.103058.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.103058.log.gz 2024-04-23T22:00:12.309 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77505.log 2024-04-23T22:00:12.312 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34519.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34519.log.gz 2024-04-23T22:00:12.316 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86752.log 2024-04-23T22:00:12.316 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77505.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70517.log 2024-04-23T22:00:12.317 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.77505.log.gz 2024-04-23T22:00:12.317 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86752.log: gzip -5 --verbose -- 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.86752.log.gz 2024-04-23T22:00:12.317 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster1-client.mirror.58092.log 2024-04-23T22:00:12.317 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.41070.log.gz 2024-04-23T22:00:12.317 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.70517.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.41299.log.gz 2024-04-23T22:00:12.317 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.41264.log.gz 2024-04-23T22:00:12.317 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.41555.log.gz 2024-04-23T22:00:12.317 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47990.log 2024-04-23T22:00:12.321 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.58092.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.0.42540.log 2024-04-23T22:00:12.321 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.47990.log: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.54593.log.gz 2024-04-23T22:00:12.321 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93445.log 2024-04-23T22:00:12.321 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.0.42540.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102698.log 2024-04-23T22:00:12.321 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.93445.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.93445.log.gz 2024-04-23T22:00:12.322 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63160.log 2024-04-23T22:00:12.322 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102698.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102698.log.gz 2024-04-23T22:00:12.322 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58935.log 2024-04-23T22:00:12.334 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.63160.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99802.log 2024-04-23T22:00:12.336 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58935.log: 88.7% 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.53858.log.gz 2024-04-23T22:00:12.336 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.53928.log.gz 2024-04-23T22:00:12.337 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.54348.log.gz 2024-04-23T22:00:12.339 INFO:teuthology.orchestra.run.smithi150.stderr: 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.58225.log.gz 2024-04-23T22:00:12.339 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.52807.log.gz 2024-04-23T22:00:12.340 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.52877.log.gz 2024-04-23T22:00:12.343 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.43605.log.gz 2024-04-23T22:00:12.343 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.42926.log.gz 2024-04-23T22:00:12.345 INFO:teuthology.orchestra.run.smithi150.stderr: 69.1% -- replaced with /var/log/ceph/cluster1-client.admin.35985.log.gz 2024-04-23T22:00:12.345 INFO:teuthology.orchestra.run.smithi150.stderr: 54.4% -- replaced with /var/log/ceph/cluster1-client.mirror.36002.log.gz 2024-04-23T22:00:12.346 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.37576.log.gz 2024-04-23T22:00:12.347 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.35486.log 2024-04-23T22:00:12.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99802.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.99802.log.gz 2024-04-23T22:00:12.355 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.45907.log.gz 2024-04-23T22:00:12.356 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/cluster1-client.admin.55765.log.gz 2024-04-23T22:00:12.365 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94261.log 2024-04-23T22:00:12.373 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.35486.log: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.91597.log.gz 2024-04-23T22:00:12.380 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.56400.log 2024-04-23T22:00:12.386 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94261.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94261.log.gz 2024-04-23T22:00:12.393 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.87916.log.gz 2024-04-23T22:00:12.395 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81616.log 2024-04-23T22:00:12.396 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.56400.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50138.log 2024-04-23T22:00:12.401 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.81616.log: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.81616.log.gz 2024-04-23T22:00:12.405 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.1.52568.log 2024-04-23T22:00:12.414 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50138.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.35548.log 2024-04-23T22:00:12.414 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.1.52568.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101573.log 2024-04-23T22:00:12.414 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.35548.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71594.log 2024-04-23T22:00:12.414 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101573.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.101573.log.gz 2024-04-23T22:00:12.414 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63229.log 2024-04-23T22:00:12.416 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71594.log: 92.0% -- replaced with /var/log/ceph/cluster2-client.mirror.71594.log.gz 2024-04-23T22:00:12.419 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51386.log 2024-04-23T22:00:12.419 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.63229.log: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.63229.log.gz 2024-04-23T22:00:12.419 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68545.log 2024-04-23T22:00:12.419 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.51386.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60173.log 2024-04-23T22:00:12.419 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68545.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.68545.log.gz 2024-04-23T22:00:12.421 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95284.log 2024-04-23T22:00:12.421 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60173.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66477.log 2024-04-23T22:00:12.425 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.95284.log: 91.4% -- replaced with /var/log/ceph/cluster1-client.mirror.95284.log.gz 2024-04-23T22:00:12.426 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104052.log 2024-04-23T22:00:12.432 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon cluster1.mon.a has been restored 2024-04-23T22:00:12.440 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66477.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.50623.log.gz 2024-04-23T22:00:12.440 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster1-client.mirror.50740.log.gz 2024-04-23T22:00:12.449 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63136.log 2024-04-23T22:00:12.449 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104052.log: 83.2% -- replaced with /var/log/ceph/cluster2-client.mirror.104052.log.gz 2024-04-23T22:00:12.455 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.50429.log.gz 2024-04-23T22:00:12.456 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.40232.log.gz 2024-04-23T22:00:12.461 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.39346.log 2024-04-23T22:00:12.471 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.63136.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95306.log 2024-04-23T22:00:12.471 INFO:teuthology.orchestra.run.smithi150.stderr: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.63136.log.gz 2024-04-23T22:00:12.486 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.39346.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.51386.log.gz 2024-04-23T22:00:12.489 INFO:teuthology.orchestra.run.smithi150.stderr: 80.6% -- replaced with /var/log/ceph/cluster2-client.mirror.36207.log.gz 2024-04-23T22:00:12.490 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34087.log 2024-04-23T22:00:12.491 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95306.log: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.47214.log.gz 2024-04-23T22:00:12.491 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.47796.log.gz 2024-04-23T22:00:12.492 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.47990.log.gz 2024-04-23T22:00:12.493 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.49223.log.gz 2024-04-23T22:00:12.501 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93943.log 2024-04-23T22:00:12.502 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34087.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34087.log.gz 2024-04-23T22:00:12.506 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% -- replaced with /var/log/ceph/cluster2-client.mirror.49442.log.gz 2024-04-23T22:00:12.522 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.88311.log 2024-04-23T22:00:12.522 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.93943.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.93943.log.gz 2024-04-23T22:00:12.540 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89753.log 2024-04-23T22:00:12.542 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.88311.log: 91.8% -- replaced with /var/log/ceph/cluster1-client.mirror.88311.log.gz 2024-04-23T22:00:12.558 INFO:teuthology.orchestra.run.smithi150.stderr: 95.5% 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.63160.log.gz 2024-04-23T22:00:12.560 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102721.log 2024-04-23T22:00:12.560 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89753.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.89753.log.gz 2024-04-23T22:00:12.560 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55223.log 2024-04-23T22:00:12.562 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102721.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.102721.log.gz 2024-04-23T22:00:12.564 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64180.log 2024-04-23T22:00:12.564 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55223.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82135.log 2024-04-23T22:00:12.567 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.64180.log: 93.8% -- replaced with /var/log/ceph/cluster1-client.mirror.64180.log.gz 2024-04-23T22:00:12.567 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54173.log 2024-04-23T22:00:12.567 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.82135.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.82135.log.gz 2024-04-23T22:00:12.567 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.60790.log.gz 2024-04-23T22:00:12.567 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67750.log 2024-04-23T22:00:12.568 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.54173.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99260.log 2024-04-23T22:00:12.568 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.67750.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.67750.log.gz 2024-04-23T22:00:12.569 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98206.log 2024-04-23T22:00:12.571 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99260.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.99260.log.gz 2024-04-23T22:00:12.571 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102541.log 2024-04-23T22:00:12.572 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98206.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.98206.log.gz 2024-04-23T22:00:12.576 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.94584.log 2024-04-23T22:00:12.576 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102541.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102541.log.gz 2024-04-23T22:00:12.576 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75393.log 2024-04-23T22:00:12.576 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.94584.log: 87.9% -- replaced with /var/log/ceph/cluster1-client.mirror.94584.log.gz 2024-04-23T22:00:12.577 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73105.log 2024-04-23T22:00:12.580 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75393.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.75393.log.gz 2024-04-23T22:00:12.581 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101663.log 2024-04-23T22:00:12.583 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.73105.log: 91.9% -- replaced with /var/log/ceph/cluster1-client.mirror.73105.log.gz 2024-04-23T22:00:12.584 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99902.log 2024-04-23T22:00:12.584 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101663.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101663.log.gz 2024-04-23T22:00:12.584 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104011.log 2024-04-23T22:00:12.586 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99902.log: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.99902.log.gz 2024-04-23T22:00:12.586 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97689.log 2024-04-23T22:00:12.589 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104011.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.104011.log.gz 2024-04-23T22:00:12.589 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100441.log 2024-04-23T22:00:12.594 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97689.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.97689.log.gz 2024-04-23T22:00:12.594 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46923.log 2024-04-23T22:00:12.596 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100441.log: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.70517.log.gz 2024-04-23T22:00:12.596 INFO:teuthology.orchestra.run.smithi150.stderr: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.57943.log.gz 2024-04-23T22:00:12.596 INFO:teuthology.orchestra.run.smithi150.stderr: 90.6% -- replaced with /var/log/ceph/cluster2-client.mirror.100441.log.gz 2024-04-23T22:00:12.601 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34311.log 2024-04-23T22:00:12.602 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46923.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100186.log 2024-04-23T22:00:12.602 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34311.log: 0.0%gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57877.log 2024-04-23T22:00:12.602 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.admin.34311.log.gz 2024-04-23T22:00:12.602 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.44526.log 2024-04-23T22:00:12.602 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100186.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85535.log 2024-04-23T22:00:12.602 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57877.log: /var/log/ceph/cluster1-client.mirror.44526.log: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.100186.log.gz 2024-04-23T22:00:12.602 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.0.78843.log 2024-04-23T22:00:12.602 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85535.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.85535.log.gz 2024-04-23T22:00:12.602 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53297.log 2024-04-23T22:00:12.602 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.0.78843.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73356.log 2024-04-23T22:00:12.603 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.53297.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86914.log 2024-04-23T22:00:12.604 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73356.log: 92.2% -- replaced with /var/log/ceph/cluster2-client.mirror.73356.log.gz 2024-04-23T22:00:12.607 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102361.log 2024-04-23T22:00:12.608 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86914.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.86914.log.gz 2024-04-23T22:00:12.609 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.36294.log 2024-04-23T22:00:12.609 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102361.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102361.log.gz 2024-04-23T22:00:12.611 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34014.log 2024-04-23T22:00:12.611 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.36294.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53514.log 2024-04-23T22:00:12.611 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34014.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34014.log.gz 2024-04-23T22:00:12.611 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100066.log 2024-04-23T22:00:12.612 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.53514.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48669.log 2024-04-23T22:00:12.613 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100066.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.100066.log.gz 2024-04-23T22:00:12.615 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70277.log 2024-04-23T22:00:12.616 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48669.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41881.log 2024-04-23T22:00:12.619 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.70277.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.70277.log.gz 2024-04-23T22:00:12.620 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53753.log 2024-04-23T22:00:12.620 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41881.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88118.log 2024-04-23T22:00:12.621 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.41943.log.gz 2024-04-23T22:00:12.623 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.41881.log.gz 2024-04-23T22:00:12.624 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.53753.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101236.log 2024-04-23T22:00:12.626 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88118.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.88118.log.gz 2024-04-23T22:00:12.626 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101483.log 2024-04-23T22:00:12.627 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101236.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101236.log.gz 2024-04-23T22:00:12.627 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42670.log 2024-04-23T22:00:12.629 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.55223.log.gz 2024-04-23T22:00:12.629 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.57513.log.gz 2024-04-23T22:00:12.630 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101483.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.101483.log.gz 2024-04-23T22:00:12.630 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82671.log 2024-04-23T22:00:12.633 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.42670.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95001.log 2024-04-23T22:00:12.635 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.82671.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.82671.log.gz 2024-04-23T22:00:12.635 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.107278.log 2024-04-23T22:00:12.637 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95001.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.95001.log.gz 93.1% 2024-04-23T22:00:12.637 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.57748.log.gz 2024-04-23T22:00:12.637 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65303.log 2024-04-23T22:00:12.637 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.107278.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.107278.log.gz 2024-04-23T22:00:12.638 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58460.log 2024-04-23T22:00:12.643 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.65303.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.65303.log.gz 2024-04-23T22:00:12.643 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.106169.log.gz 2024-04-23T22:00:12.643 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster2-client.mirror.48052.log 2024-04-23T22:00:12.643 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61341.log 2024-04-23T22:00:12.643 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58460.log: /var/log/ceph/cluster2-client.mirror.48052.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102518.log 2024-04-23T22:00:12.643 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.61341.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36036.log 2024-04-23T22:00:12.643 INFO:teuthology.orchestra.run.smithi150.stderr: 84.5% -- replaced with /var/log/ceph/cluster2-client.mirror.61341.log.gz 2024-04-23T22:00:12.643 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102518.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49485.log 2024-04-23T22:00:12.643 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.36036.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102518.log.gz 2024-04-23T22:00:12.644 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100015.log 2024-04-23T22:00:12.644 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.49485.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.0.63047.log 2024-04-23T22:00:12.644 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100015.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95652.log 2024-04-23T22:00:12.646 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.0.63047.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.100015.log.gz 2024-04-23T22:00:12.646 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102181.log 2024-04-23T22:00:12.646 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95652.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.95652.log.gz 2024-04-23T22:00:12.646 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65485.log 2024-04-23T22:00:12.646 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102181.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102181.log.gz 2024-04-23T22:00:12.647 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47179.log 2024-04-23T22:00:12.651 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.65485.log: 91.6% 94.1% -- replaced with /var/log/ceph/cluster2-client.mirror.0.78843.log.gz 2024-04-23T22:00:12.651 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.65485.log.gz 2024-04-23T22:00:12.651 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106630.log 2024-04-23T22:00:12.651 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.47179.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70366.log 2024-04-23T22:00:12.651 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57705.log 2024-04-23T22:00:12.651 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.70366.log: /var/log/ceph/cluster1-client.mirror.106630.log: 87.9% 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.106630.log.gz 2024-04-23T22:00:12.651 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.70366.log.gz 2024-04-23T22:00:12.651 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45296.log 2024-04-23T22:00:12.656 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70656.log 2024-04-23T22:00:12.656 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57705.log: /var/log/ceph/cluster2-client.mirror.45296.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84359.log 2024-04-23T22:00:12.656 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.0.49691.log 2024-04-23T22:00:12.656 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.70656.log: /var/log/ceph/cluster2-client.mirror.84359.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.70656.log.gz 2024-04-23T22:00:12.657 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.84359.log.gz 2024-04-23T22:00:12.657 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67381.log 2024-04-23T22:00:12.657 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101078.log 2024-04-23T22:00:12.657 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.67381.log: /var/log/ceph/cluster2-client.mirror.0.49691.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101843.log 2024-04-23T22:00:12.657 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.67381.log.gz 2024-04-23T22:00:12.657 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101078.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59844.log 2024-04-23T22:00:12.657 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101843.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101078.log.gz 2024-04-23T22:00:12.657 INFO:teuthology.orchestra.run.smithi150.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.101843.log.gz 2024-04-23T22:00:12.657 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72908.log 2024-04-23T22:00:12.657 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster2-client.mirror.59844.log: -- /var/log/ceph/cluster2-client.mirror.84483.log 2024-04-23T22:00:12.657 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.37245.log 2024-04-23T22:00:12.657 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4%/var/log/ceph/cluster2-client.mirror.84483.log: -- replaced with /var/log/ceph/cluster2-client.mirror.46923.log.gz 2024-04-23T22:00:12.657 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.72908.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.84483.log.gz 2024-04-23T22:00:12.666 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91068.log 2024-04-23T22:00:12.666 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.72908.log.gz 2024-04-23T22:00:12.666 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.37245.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106858.log 2024-04-23T22:00:12.666 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.91068.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105404.log 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.91068.log.gz 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.106858.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106858.log.gzgzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57002.log 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.105404.log: gzip 86.7% -5 --verbose -- /var/log/ceph/cluster2-client.mirror.1.53514.log 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.105404.log.gz 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57002.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43252.log 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.1.53514.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43314.log 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose/var/log/ceph/cluster2-client.mirror.43252.log: -- /var/log/ceph/cluster1-client.mirror.79324.log 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.35151.log 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.79324.log: /var/log/ceph/cluster2-client.mirror.43314.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103230.log 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr: 92.0% -- replaced with /var/log/ceph/cluster1-client.mirror.79324.log.gz 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.35151.log: /var/log/ceph/cluster2-client.mirror.103230.log: 83.4% -- replaced with /var/log/ceph/cluster2-client.mirror.103230.log.gz 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.35151.log.gz 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99564.log 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76537.log 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose --/var/log/ceph/cluster2-client.mirror.99564.log: /var/log/ceph/cluster2-client.mirror.89202.log 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.66477.log.gz 2024-04-23T22:00:12.667 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.668 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76537.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47311.log 2024-04-23T22:00:12.669 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89202.log: 90.4% -- replaced with /var/log/ceph/cluster2-client.mirror.99564.log.gz 2024-04-23T22:00:12.669 INFO:teuthology.orchestra.run.smithi150.stderr: 92.6% -- replaced with /var/log/ceph/cluster2-client.mirror.0.42540.log.gz 2024-04-23T22:00:12.669 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.3.74185.log 2024-04-23T22:00:12.669 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.47311.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46826.log 2024-04-23T22:00:12.675 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.3.74185.log: 94.2% -- replaced with /var/log/ceph/cluster2-client.mirror.0.63047.log.gz 2024-04-23T22:00:12.676 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57364.log 2024-04-23T22:00:12.677 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6%/var/log/ceph/cluster2-client.mirror.46826.log: -- replaced with /var/log/ceph/cluster2-client.mirror.83997.log.gz 2024-04-23T22:00:12.679 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44333.log 2024-04-23T22:00:12.679 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57364.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.55734.log 2024-04-23T22:00:12.679 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44333.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101371.log 2024-04-23T22:00:12.679 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87414.log 2024-04-23T22:00:12.680 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101371.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101371.log.gz 2024-04-23T22:00:12.681 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.55734.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43446.log 2024-04-23T22:00:12.683 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87414.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.37083.log 2024-04-23T22:00:12.683 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.43446.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.87414.log.gz 2024-04-23T22:00:12.683 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42075.log 2024-04-23T22:00:12.683 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.37083.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.92742.log 2024-04-23T22:00:12.683 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57235.log 2024-04-23T22:00:12.685 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.42075.log: /var/log/ceph/cluster1-client.mirror.92742.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101438.log 2024-04-23T22:00:12.686 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57235.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89068.log 2024-04-23T22:00:12.686 INFO:teuthology.orchestra.run.smithi150.stderr: 91.7% -- replaced with /var/log/ceph/cluster1-client.mirror.92742.log.gz 2024-04-23T22:00:12.688 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.1.103348.log/var/log/ceph/cluster2-client.mirror.89068.log: 2024-04-23T22:00:12.688 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101438.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.89068.log.gz 2024-04-23T22:00:12.688 INFO:teuthology.orchestra.run.smithi150.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.101438.log.gz 2024-04-23T22:00:12.688 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88494.log 2024-04-23T22:00:12.691 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.1.103348.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.3.51781.log 2024-04-23T22:00:12.694 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88494.log: 91.9% -- replaced with /var/log/ceph/cluster2-client.mirror.1.103348.log.gz 2024-04-23T22:00:12.694 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.88494.log.gz 2024-04-23T22:00:12.694 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49812.log 2024-04-23T22:00:12.701 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.3.51781.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46285.log 2024-04-23T22:00:12.701 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86109.log 2024-04-23T22:00:12.701 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46285.log: /var/log/ceph/cluster2-client.mirror.49812.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94517.log 2024-04-23T22:00:12.702 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84912.log 2024-04-23T22:00:12.702 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94517.log: /var/log/ceph/cluster2-client.mirror.86109.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94517.log.gz 2024-04-23T22:00:12.702 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.86109.log.gz 2024-04-23T22:00:12.702 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.104629.log 2024-04-23T22:00:12.702 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.84912.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102833.log 2024-04-23T22:00:12.702 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.104629.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.104629.log.gz 2024-04-23T22:00:12.702 INFO:teuthology.orchestra.run.smithi150.stderr: 94.0% -- replaced with /var/log/ceph/cluster1-client.mirror.3.62975.log.gz 2024-04-23T22:00:12.702 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93978.log 2024-04-23T22:00:12.702 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102833.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82250.log 2024-04-23T22:00:12.702 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102833.log.gz 2024-04-23T22:00:12.702 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.93978.log: 87.1% -- replaced with /var/log/ceph/cluster2-client.mirror.93978.log.gz 2024-04-23T22:00:12.702 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94609.log 2024-04-23T22:00:12.702 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.82250.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80817.log 2024-04-23T22:00:12.703 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82250.log.gz 2024-04-23T22:00:12.703 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94609.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50704.log 2024-04-23T22:00:12.703 INFO:teuthology.orchestra.run.smithi150.stderr: 92.0% -- replaced with /var/log/ceph/cluster2-client.mirror.94609.log.gz 2024-04-23T22:00:12.704 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.80817.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79705.log 2024-04-23T22:00:12.704 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50704.log: gzip -5 --verbose -- /var/log/ceph/cluster2.log 2024-04-23T22:00:12.704 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.79705.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.79705.log.gz 2024-04-23T22:00:12.706 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52281.log 2024-04-23T22:00:12.715 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69517.log 2024-04-23T22:00:12.719 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52281.log: 87.3% -- replaced with /var/log/ceph/cluster2.log.gz 2024-04-23T22:00:12.719 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58136.log 2024-04-23T22:00:12.720 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.69517.log: 87.9% -- replaced with /var/log/ceph/cluster1-client.mirror.69517.log.gz 2024-04-23T22:00:12.720 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59421.log 2024-04-23T22:00:12.726 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58136.log: /var/log/ceph/cluster2-client.mirror.59421.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85075.log 2024-04-23T22:00:12.726 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55865.log 2024-04-23T22:00:12.727 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85075.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.85075.log.gz 2024-04-23T22:00:12.727 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52084.log 2024-04-23T22:00:12.729 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55865.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102406.log 2024-04-23T22:00:12.729 INFO:teuthology.orchestra.run.smithi150.stderr: 94.0% -- replaced with /var/log/ceph/cluster2-client.mirror.3.74185.log.gz 2024-04-23T22:00:12.734 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52084.log: /var/log/ceph/cluster1-client.mirror.102406.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102406.log.gz 2024-04-23T22:00:12.734 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51443.log 2024-04-23T22:00:12.734 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55118.log 2024-04-23T22:00:12.734 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.51443.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55595.log 2024-04-23T22:00:12.734 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55118.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89531.log 2024-04-23T22:00:12.734 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.55595.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.45689.log 2024-04-23T22:00:12.734 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89531.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.2.91942.log 2024-04-23T22:00:12.734 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.89531.log.gz 2024-04-23T22:00:12.734 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.45689.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71040.log 2024-04-23T22:00:12.740 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.2.91942.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106883.log 2024-04-23T22:00:12.743 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.71040.log: 91.7% -- replaced with /var/log/ceph/cluster1-client.mirror.71040.log.gz 2024-04-23T22:00:12.743 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.50704.log.gz 2024-04-23T22:00:12.744 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/cluster1-client.admin.50926.log.gz 2024-04-23T22:00:12.746 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45493.log 2024-04-23T22:00:12.746 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.106883.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106883.log.gz 2024-04-23T22:00:12.746 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86671.log 2024-04-23T22:00:12.748 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.45493.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99428.log 2024-04-23T22:00:12.750 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86671.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.86671.log.gz 2024-04-23T22:00:12.750 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61364.log 2024-04-23T22:00:12.751 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99428.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.40003.log.gz 2024-04-23T22:00:12.753 INFO:teuthology.orchestra.run.smithi150.stderr: 82.9% -- replaced with /var/log/ceph/cluster1-client.mirror.53514.log.gz 2024-04-23T22:00:12.753 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.53753.log.gz 2024-04-23T22:00:12.756 INFO:teuthology.orchestra.run.smithi150.stderr: 90.5% 96.6% -- replaced with /var/log/ceph/cluster2-client.mirror.51608.log.gz 2024-04-23T22:00:12.756 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% -- replaced with /var/log/ceph/cluster2-client.mirror.44591.log.gz 2024-04-23T22:00:12.757 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.42075.log.gz 2024-04-23T22:00:12.757 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.42670.log.gz 2024-04-23T22:00:12.757 INFO:teuthology.orchestra.run.smithi150.stderr: 89.5% -- replaced with /var/log/ceph/cluster2-client.mirror.58136.log.gz 2024-04-23T22:00:12.757 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.99428.log.gz 2024-04-23T22:00:12.758 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58292.log 2024-04-23T22:00:12.758 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.43314.log.gz 2024-04-23T22:00:12.760 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.61364.log: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.76537.log.gz 2024-04-23T22:00:12.761 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102923.log 2024-04-23T22:00:12.761 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.58292.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.31281.log 2024-04-23T22:00:12.763 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102923.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102923.log.gz 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.58460.log.gz 2024-04-23T22:00:12.763 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.763 INFO:teuthology.orchestra.run.smithi150.stderr: 73.4% -- replaced with /var/log/ceph/cluster1-client.mirror.37245.log.gzgzip 2024-04-23T22:00:12.763 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.75662.log 2024-04-23T22:00:12.763 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.31281.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.31281.log.gz 2024-04-23T22:00:12.764 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.55433.log.gz 2024-04-23T22:00:12.765 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.48052.log.gz 2024-04-23T22:00:12.767 INFO:teuthology.orchestra.run.smithi150.stderr: 76.3% -- replaced with /var/log/ceph/cluster1-client.mirror.45689.log.gz 2024-04-23T22:00:12.770 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73485.log 2024-04-23T22:00:12.773 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.75662.log: 87.9% -- replaced with /var/log/ceph/cluster1-client.mirror.75662.log.gz 2024-04-23T22:00:12.775 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77292.log 2024-04-23T22:00:12.775 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.73485.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.73485.log.gz 2024-04-23T22:00:12.775 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50200.log 2024-04-23T22:00:12.775 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77292.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.77292.log.gz 2024-04-23T22:00:12.778 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.39387.log 2024-04-23T22:00:12.778 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.50200.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.58621.log 2024-04-23T22:00:12.778 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.39387.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62401.log 2024-04-23T22:00:12.778 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.58621.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48087.log 2024-04-23T22:00:12.785 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.62401.log: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.89202.log.gz 2024-04-23T22:00:12.786 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79829.log 2024-04-23T22:00:12.788 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48087.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.35235.log 2024-04-23T22:00:12.789 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.79829.log: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.2.91942.log.gz 2024-04-23T22:00:12.789 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.79829.log.gz 2024-04-23T22:00:12.789 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.37475.log 2024-04-23T22:00:12.802 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.35235.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100563.log 2024-04-23T22:00:12.802 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.35235.log.gz 2024-04-23T22:00:12.802 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.37475.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71885.log 2024-04-23T22:00:12.802 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100563.log: gzip -5 --verbose -- /var/log/ceph/cluster2.audit.log 2024-04-23T22:00:12.802 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71885.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.100563.log.gz 2024-04-23T22:00:12.802 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.95306.log.gz 2024-04-23T22:00:12.802 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.71885.log.gz 2024-04-23T22:00:12.802 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54243.log 2024-04-23T22:00:12.802 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2.audit.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98798.log 2024-04-23T22:00:12.802 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.54243.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57986.log 2024-04-23T22:00:12.803 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82295.log 2024-04-23T22:00:12.803 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98798.log: /var/log/ceph/cluster2-client.mirror.57986.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68250.log 2024-04-23T22:00:12.803 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.82295.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38999.log 2024-04-23T22:00:12.803 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68250.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.98798.log.gz 2024-04-23T22:00:12.803 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.68250.log.gz 2024-04-23T22:00:12.803 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69792.log 2024-04-23T22:00:12.803 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98646.log 2024-04-23T22:00:12.803 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69792.log: /var/log/ceph/cluster2-client.mirror.38999.log: 92.0% -- replaced with /var/log/ceph/cluster1-client.mirror.82295.log.gz 2024-04-23T22:00:12.803 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.69792.log.gz 2024-04-23T22:00:12.803 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.0.62008.log 2024-04-23T22:00:12.803 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.80817.log.gz 2024-04-23T22:00:12.803 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.37030.log 2024-04-23T22:00:12.803 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98646.log: /var/log/ceph/cluster2-client.mirror.0.62008.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.48184.log 2024-04-23T22:00:12.803 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.37030.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46069.log 2024-04-23T22:00:12.803 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.48184.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.44125.log 2024-04-23T22:00:12.804 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46069.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77537.log 2024-04-23T22:00:12.804 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.44125.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105966.log 2024-04-23T22:00:12.804 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.77537.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100370.log 2024-04-23T22:00:12.804 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105966.log: 90.6% 87.9% -- replaced with /var/log/ceph/cluster1-client.mirror.77537.log.gz 2024-04-23T22:00:12.804 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.98646.log.gz 2024-04-23T22:00:12.804 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77735.log 2024-04-23T22:00:12.804 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100370.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57728.log 2024-04-23T22:00:12.804 INFO:teuthology.orchestra.run.smithi150.stderr: 91.3% -- replaced with /var/log/ceph/cluster2-client.mirror.105966.log.gz 2024-04-23T22:00:12.804 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77735.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.77735.log.gz 2024-04-23T22:00:12.804 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86868.log 2024-04-23T22:00:12.804 INFO:teuthology.orchestra.run.smithi150.stderr: 90.5%/var/log/ceph/cluster2-client.mirror.57728.log: -- replaced with /var/log/ceph/cluster2-client.mirror.100370.log.gz 2024-04-23T22:00:12.804 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56272.log 2024-04-23T22:00:12.804 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.34771.log 2024-04-23T22:00:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86868.log: /var/log/ceph/cluster2-client.mirror.56272.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.93537.log 2024-04-23T22:00:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.34771.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.34771.log.gz 2024-04-23T22:00:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70928.log 2024-04-23T22:00:12.819 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.86868.log.gz/var/log/ceph/cluster2-client.mirror.93537.log: 2024-04-23T22:00:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105217.log 2024-04-23T22:00:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.70928.log: gzip -5 --verbose -- 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.70928.log.gz 2024-04-23T22:00:12.819 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster2-client.mirror.96128.log 2024-04-23T22:00:12.819 INFO:teuthology.orchestra.run.smithi150.stderr: 93.5% -- replaced with /var/log/ceph/cluster2-client.mirror.0.62008.log.gz 2024-04-23T22:00:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71197.log 2024-04-23T22:00:12.819 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105217.log: /var/log/ceph/cluster2-client.mirror.96128.log: gzip -5 --verbose -- 92.3% /var/log/ceph/cluster2-client.mirror.47373.log 2024-04-23T22:00:12.820 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.93537.log.gz 2024-04-23T22:00:12.820 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.71197.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.96128.log.gz 2024-04-23T22:00:12.820 INFO:teuthology.orchestra.run.smithi150.stderr: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.105217.log.gz 2024-04-23T22:00:12.820 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.71197.log.gz 2024-04-23T22:00:12.820 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105299.log 2024-04-23T22:00:12.820 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.47373.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103715.log 2024-04-23T22:00:12.820 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40876.log 2024-04-23T22:00:12.820 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.103715.log: /var/log/ceph/cluster1-client.mirror.105299.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.107488.log 2024-04-23T22:00:12.820 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40876.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55658.log 2024-04-23T22:00:12.820 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.107488.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.107488.log.gz 2024-04-23T22:00:12.820 INFO:teuthology.orchestra.run.smithi150.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.105299.log.gz 2024-04-23T22:00:12.820 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97381.log 2024-04-23T22:00:12.820 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.55658.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.0.35576.log 2024-04-23T22:00:12.820 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97381.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.35627.log 2024-04-23T22:00:12.820 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.97381.log.gz 2024-04-23T22:00:12.821 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.0.35576.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76815.log 2024-04-23T22:00:12.821 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66455.log 2024-04-23T22:00:12.821 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.35627.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.39234.log 2024-04-23T22:00:12.821 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.66455.log: /var/log/ceph/cluster2-client.mirror.76815.log: gzip -5 --verbose -- 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.76815.log.gz 2024-04-23T22:00:12.821 INFO:teuthology.orchestra.run.smithi150.stderr: /var/log/ceph/cluster2-client.mirror.106467.log 2024-04-23T22:00:12.821 INFO:teuthology.orchestra.run.smithi150.stderr: 91.8% -- replaced with /var/log/ceph/cluster1-client.mirror.66455.log.gz 2024-04-23T22:00:12.821 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60610.log 2024-04-23T22:00:12.821 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106467.log: /var/log/ceph/cluster2-client.mirror.39234.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99873.log 2024-04-23T22:00:12.821 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60610.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.2.83515.log 2024-04-23T22:00:12.821 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99873.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52200.log 2024-04-23T22:00:12.821 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.2.83515.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.99873.log.gz 2024-04-23T22:00:12.821 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100776.log 2024-04-23T22:00:12.821 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52200.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97110.log 2024-04-23T22:00:12.823 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% -- replaced with /var/log/ceph/cluster1-client.mirror.52358.log.gz 2024-04-23T22:00:12.823 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100776.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.100776.log.gz 2024-04-23T22:00:12.823 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57642.log 2024-04-23T22:00:12.830 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97110.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.96736.log 2024-04-23T22:00:12.832 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.57642.log: 90.6% -- replaced with /var/log/ceph/cluster2-client.mirror.97110.log.gz 2024-04-23T22:00:12.834 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67785.log 2024-04-23T22:00:12.836 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.96736.log: 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.96736.log.gz 2024-04-23T22:00:12.836 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86544.log 2024-04-23T22:00:12.837 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.67785.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.67785.log.gz 2024-04-23T22:00:12.837 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46985.log 2024-04-23T22:00:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86544.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.86544.log.gz 2024-04-23T22:00:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96849.log 2024-04-23T22:00:12.839 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46985.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105674.log 2024-04-23T22:00:12.841 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.96849.log: 90.6% -- replaced with /var/log/ceph/cluster2-client.mirror.96849.log.gz 2024-04-23T22:00:12.843 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59777.log 2024-04-23T22:00:12.847 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.105674.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.105674.log.gz 2024-04-23T22:00:12.847 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46791.log 2024-04-23T22:00:12.847 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.59777.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.103627.log 2024-04-23T22:00:12.847 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46791.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54068.log 2024-04-23T22:00:12.849 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.103627.log: 91.4% -- replaced with /var/log/ceph/cluster2-client.mirror.103627.log.gz 2024-04-23T22:00:12.851 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45199.log 2024-04-23T22:00:12.851 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.54068.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88926.log 2024-04-23T22:00:12.851 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.45199.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77845.log 2024-04-23T22:00:12.852 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88926.log: 93.8% -- replaced with /var/log/ceph/cluster2-client.mirror.2.83515.log.gz 2024-04-23T22:00:12.853 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.46023.log 2024-04-23T22:00:12.853 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77845.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.77845.log.gz 2024-04-23T22:00:12.855 INFO:teuthology.orchestra.run.smithi150.stderr: 92.0% -- replaced with /var/log/ceph/cluster2-client.mirror.88926.log.gz 2024-04-23T22:00:12.856 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54488.log 2024-04-23T22:00:12.856 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.46023.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82938.log 2024-04-23T22:00:12.856 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.54488.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89160.log 2024-04-23T22:00:12.858 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.82938.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.82938.log.gz 2024-04-23T22:00:12.859 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75162.log 2024-04-23T22:00:12.859 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89160.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52003.log 2024-04-23T22:00:12.859 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.75162.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.75162.log.gz 2024-04-23T22:00:12.859 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67567.log 2024-04-23T22:00:12.859 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52003.log: 92.1% -- replaced with /var/log/ceph/cluster2-client.mirror.89160.log.gz 2024-04-23T22:00:12.859 INFO:teuthology.orchestra.run.smithi150.stderr: 4.7% -- replaced with /var/log/ceph/cluster1-client.mirror.36036.log.gz 2024-04-23T22:00:12.864 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38626.log 2024-04-23T22:00:12.864 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.67567.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74689.log 2024-04-23T22:00:12.864 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.67567.log.gz 2024-04-23T22:00:12.864 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.38626.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69313.log 2024-04-23T22:00:12.864 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74689.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66181.log 2024-04-23T22:00:12.864 INFO:teuthology.orchestra.run.smithi150.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.74689.log.gz 2024-04-23T22:00:12.864 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69313.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.69313.log.gz 2024-04-23T22:00:12.864 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.54488.log.gz 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.54453.log.gz 2024-04-23T22:00:12.864 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.869 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85009.log 2024-04-23T22:00:12.869 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.66181.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.66181.log.gz 2024-04-23T22:00:12.869 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81396.log 2024-04-23T22:00:12.869 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.85009.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53402.log 2024-04-23T22:00:12.869 INFO:teuthology.orchestra.run.smithi150.stderr: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.85009.log.gz 2024-04-23T22:00:12.869 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81396.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.36104.log 2024-04-23T22:00:12.869 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.53402.log: 93.1% -- replaced with /var/log/ceph/cluster2-client.mirror.51486.log.gz 2024-04-23T22:00:12.869 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90844.log 2024-04-23T22:00:12.869 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.36104.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.2.39387.log 2024-04-23T22:00:12.869 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90844.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.90844.log.gz 2024-04-23T22:00:12.871 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96229.log 2024-04-23T22:00:12.871 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.2.39387.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105759.log 2024-04-23T22:00:12.871 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.96229.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.96229.log.gz 2024-04-23T22:00:12.872 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106102.log 2024-04-23T22:00:12.874 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105759.log: 91.4% -- replaced with /var/log/ceph/cluster2-client.mirror.105759.log.gz 2024-04-23T22:00:12.878 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77203.log 2024-04-23T22:00:12.878 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106102.log: 83.2% -- replaced with /var/log/ceph/cluster2-client.mirror.106102.log.gz 2024-04-23T22:00:12.878 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.45261.log 2024-04-23T22:00:12.878 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77203.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.77203.log.gz 2024-04-23T22:00:12.883 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.42269.log 2024-04-23T22:00:12.883 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.45261.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51351.log 2024-04-23T22:00:12.883 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.42269.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69170.log 2024-04-23T22:00:12.883 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.51351.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72690.log 2024-04-23T22:00:12.883 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69170.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69170.log.gz 2024-04-23T22:00:12.883 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62824.log 2024-04-23T22:00:12.883 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.72690.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.72690.log.gz 2024-04-23T22:00:12.883 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.39318.log 2024-04-23T22:00:12.883 INFO:teuthology.orchestra.run.smithi150.stderr: 80.0% -- replaced with /var/log/ceph/cluster2-client.admin.36294.log.gz 2024-04-23T22:00:12.884 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.62824.log: 84.7% -- replaced with /var/log/ceph/cluster2-client.mirror.62824.log.gz 2024-04-23T22:00:12.885 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98989.log 2024-04-23T22:00:12.885 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.42269.log.gz 2024-04-23T22:00:12.886 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.admin.39318.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101618.log 2024-04-23T22:00:12.887 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98989.log: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.98989.log.gz 2024-04-23T22:00:12.887 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.43446.log.gz 2024-04-23T22:00:12.890 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101933.log 2024-04-23T22:00:12.891 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101618.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101618.log.gz 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.58292.log.gz 2024-04-23T22:00:12.891 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.892 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.55118.log.gz 2024-04-23T22:00:12.892 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/cluster2-client.mirror.40488.log 2024-04-23T22:00:12.892 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101933.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101933.log.gz 2024-04-23T22:00:12.903 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56878.log 2024-04-23T22:00:12.903 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40488.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102001.log 2024-04-23T22:00:12.903 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56878.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.105914.log 2024-04-23T22:00:12.903 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.38626.log.gz 2024-04-23T22:00:12.903 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102001.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.102001.log.gz 2024-04-23T22:00:12.903 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.38933.log.gz 2024-04-23T22:00:12.903 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.47373.log.gz 2024-04-23T22:00:12.904 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.48184.log.gz 2024-04-23T22:00:12.904 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster1-client.admin.39318.log.gz 2024-04-23T22:00:12.904 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.45261.log.gz -- replaced with /var/log/ceph/cluster2-client.mirror.46069.log.gz 2024-04-23T22:00:12.904 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.904 INFO:teuthology.orchestra.run.smithi150.stderr: 81.8% -- replaced with /var/log/ceph/cluster2-client.mirror.39387.log.gz 2024-04-23T22:00:12.904 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.45199.log.gz 2024-04-23T22:00:12.904 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102496.log 2024-04-23T22:00:12.904 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.105914.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.105914.log.gz 2024-04-23T22:00:12.904 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.95490.log 2024-04-23T22:00:12.904 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102496.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102496.log.gz 2024-04-23T22:00:12.904 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54873.log 2024-04-23T22:00:12.904 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.95490.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.95490.log.gz 2024-04-23T22:00:12.905 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81297.log/var/log/ceph/cluster2-client.mirror.54873.log: 2024-04-23T22:00:12.905 INFO:teuthology.orchestra.run.smithi150.stderr: 93.2% -- replaced with /var/log/ceph/cluster1-client.mirror.57877.log.gz 2024-04-23T22:00:12.907 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.95953.log 2024-04-23T22:00:12.907 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81297.log: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.81297.log.gz 2024-04-23T22:00:12.910 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.89855.log 2024-04-23T22:00:12.911 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.95953.log: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.95953.log.gz 2024-04-23T22:00:12.912 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.94651.log 2024-04-23T22:00:12.912 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.89855.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.89855.log.gz 2024-04-23T22:00:12.912 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.91365.log 2024-04-23T22:00:12.923 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.94651.log: gzip 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.49304.log.gz 2024-04-23T22:00:12.923 INFO:teuthology.orchestra.run.smithi150.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83050.log 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.49385.log.gz 2024-04-23T22:00:12.923 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:12.924 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.91365.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.91365.log.gz 2024-04-23T22:00:12.925 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.52982.log.gz 2024-04-23T22:00:12.942 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59446.log 2024-04-23T22:00:12.943 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.83050.log: 91.8% -- replaced with /var/log/ceph/cluster1-client.mirror.83050.log.gz 2024-04-23T22:00:12.944 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61807.log 2024-04-23T22:00:12.953 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.59446.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102653.log 2024-04-23T22:00:12.957 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.61807.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.52084.log.gz 2024-04-23T22:00:12.957 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.52281.log.gz 2024-04-23T22:00:12.962 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80375.log 2024-04-23T22:00:12.966 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102653.log: 92.7% -- replaced with /var/log/ceph/cluster1-client.mirror.45470.log.gz 2024-04-23T22:00:12.966 INFO:teuthology.orchestra.run.smithi150.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.102653.log.gz 2024-04-23T22:00:12.972 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.50006.log.gz 2024-04-23T22:00:12.972 INFO:teuthology.orchestra.run.smithi150.stderr: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.50235.log.gz 2024-04-23T22:00:12.972 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.50297.log.gz 2024-04-23T22:00:12.973 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.50332.log.gz 2024-04-23T22:00:12.974 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.53402.log.gz 2024-04-23T22:00:12.974 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.50138.log.gz 2024-04-23T22:00:12.974 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.54243.log.gz 2024-04-23T22:00:12.975 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.43252.log.gz 2024-04-23T22:00:12.978 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43349.log 2024-04-23T22:00:12.979 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.80375.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.80375.log.gz 2024-04-23T22:00:12.979 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.56272.log.gz 2024-04-23T22:00:12.979 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.58621.log.gz 2024-04-23T22:00:12.980 INFO:teuthology.orchestra.run.smithi150.stderr: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.53297.log.gz 2024-04-23T22:00:12.983 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.38999.log.gz 2024-04-23T22:00:12.988 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.47179.log.gz 2024-04-23T22:00:12.990 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.48669.log.gz 2024-04-23T22:00:12.995 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.104562.log 2024-04-23T22:00:13.003 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.43349.log: 93.2% -- replaced with /var/log/ceph/cluster1-client.mirror.57920.log.gz 2024-04-23T22:00:13.009 INFO:teuthology.orchestra.run.smithi150.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.57002.log.gz 2024-04-23T22:00:13.009 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62104.log 2024-04-23T22:00:13.016 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.104562.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77098.log 2024-04-23T22:00:13.021 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.62104.log: 84.5% -- replaced with /var/log/ceph/cluster2-client.mirror.62104.log.gz 2024-04-23T22:00:13.022 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.49812.log.gz 2024-04-23T22:00:13.027 INFO:teuthology.orchestra.run.smithi150.stderr: 95.5% -- replaced with /var/log/ceph/cluster2-client.mirror.106467.log.gz 2024-04-23T22:00:13.035 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.62401.log.gz 2024-04-23T22:00:13.037 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64112.log 2024-04-23T22:00:13.038 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77098.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.77098.log.gz 2024-04-23T22:00:13.050 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62847.log 2024-04-23T22:00:13.051 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.64112.log: 84.3% -- replaced with /var/log/ceph/cluster2-client.mirror.64112.log.gz 2024-04-23T22:00:13.054 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.81396.log.gz 2024-04-23T22:00:13.063 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.1.59484.log 2024-04-23T22:00:13.082 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.62847.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100796.log 2024-04-23T22:00:13.088 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.1.59484.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106040.log 2024-04-23T22:00:13.089 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.52200.log.gz 2024-04-23T22:00:13.089 INFO:teuthology.orchestra.run.smithi150.stderr: 54.5% -- replaced with /var/log/ceph/cluster1-client.mirror.36104.log.gz 2024-04-23T22:00:13.089 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.52003.log.gz 2024-04-23T22:00:13.090 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.37475.log.gz 2024-04-23T22:00:13.092 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.54068.log.gz 2024-04-23T22:00:13.103 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.87379.log 2024-04-23T22:00:13.103 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106040.log: /var/log/ceph/cluster2-client.mirror.100796.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.106040.log.gz 2024-04-23T22:00:13.103 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% -- replaced with /var/log/ceph/cluster1-client.mirror.57705.log.gz 2024-04-23T22:00:13.103 INFO:teuthology.orchestra.run.smithi150.stderr: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.100796.log.gz 2024-04-23T22:00:13.111 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.43349.log.gz 2024-04-23T22:00:13.114 INFO:teuthology.orchestra.run.smithi150.stderr: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.51351.log.gz 2024-04-23T22:00:13.116 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53648.log 2024-04-23T22:00:13.116 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.87379.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.87379.log.gz 2024-04-23T22:00:13.121 INFO:teuthology.orchestra.run.smithi150.stderr: 89.4% -- replaced with /var/log/ceph/cluster2-client.mirror.35486.log.gz 2024-04-23T22:00:13.122 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.39941.log.gz 2024-04-23T22:00:13.128 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90987.log 2024-04-23T22:00:13.129 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.53648.log: 88.1% -- replaced with /var/log/ceph/cluster1-client.admin.37083.log.gz 2024-04-23T22:00:13.130 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.54873.log.gz 2024-04-23T22:00:13.132 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.84912.log.gz 2024-04-23T22:00:13.133 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60232.log 2024-04-23T22:00:13.135 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90987.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.48087.log.gz 2024-04-23T22:00:13.135 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.90987.log.gz 2024-04-23T22:00:13.135 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.106908.log 2024-04-23T22:00:13.136 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60232.log: gzip -5 88.0% --verbose -- replaced with /var/log/ceph/cluster2-client.mirror.44333.log.gz 2024-04-23T22:00:13.136 INFO:teuthology.orchestra.run.smithi150.stderr: -- /var/log/ceph/cluster1-client.mirror.57963.log 2024-04-23T22:00:13.136 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.106908.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.106908.log.gz 2024-04-23T22:00:13.139 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 88.1% --verbose -- /var/log/ceph/cluster2-client.mirror.103249.log 2024-04-23T22:00:13.139 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.46023.log.gz 2024-04-23T22:00:13.139 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.57963.log: 89.0% -- replaced with /var/log/ceph/cluster1-client.admin.55734.log.gz 2024-04-23T22:00:13.139 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101686.log 2024-04-23T22:00:13.139 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.44125.log.gz 2024-04-23T22:00:13.139 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.103249.log: 87.8% -- replaced with /var/log/ceph/cluster2-client.mirror.103249.log.gz 2024-04-23T22:00:13.140 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.45296.log.gz 2024-04-23T22:00:13.146 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.102451.log 2024-04-23T22:00:13.147 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101686.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.101686.log.gz 2024-04-23T22:00:13.148 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.98869.log 2024-04-23T22:00:13.149 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.102451.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.102451.log.gz 2024-04-23T22:00:13.149 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62669.log 2024-04-23T22:00:13.150 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.98869.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.98869.log.gz 2024-04-23T22:00:13.150 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77416.log 2024-04-23T22:00:13.157 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.62669.log: 88.7% -- replaced with /var/log/ceph/cluster1-client.mirror.56400.log.gz 2024-04-23T22:00:13.159 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-osd.0.log 2024-04-23T22:00:13.167 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.77416.log: 87.9%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68171.log 2024-04-23T22:00:13.167 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.77416.log.gz 2024-04-23T22:00:13.182 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-osd.0.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86074.log 2024-04-23T22:00:13.187 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.68171.log: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.61364.log.gz 2024-04-23T22:00:13.189 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.42540.log 2024-04-23T22:00:13.190 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86074.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.86074.log.gz 2024-04-23T22:00:13.191 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.3.67105.log 2024-04-23T22:00:13.192 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.42540.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65817.log 2024-04-23T22:00:13.200 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.3.67105.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47858.log 2024-04-23T22:00:13.203 INFO:teuthology.orchestra.run.smithi150.stderr: 94.0% -- replaced with /var/log/ceph/cluster2-client.mirror.3.67105.log.gz 2024-04-23T22:00:13.205 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.104201.log 2024-04-23T22:00:13.205 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.65817.log: /var/log/ceph/cluster2-client.mirror.47858.log: gzip -5 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.65817.log.gz 2024-04-23T22:00:13.205 INFO:teuthology.orchestra.run.smithi150.stderr: --verbose -- /var/log/ceph/cluster2-client.mirror.86292.log 2024-04-23T22:00:13.206 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.104201.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.104201.log.gz 2024-04-23T22:00:13.206 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.90084.log 2024-04-23T22:00:13.221 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86292.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81956.log 2024-04-23T22:00:13.231 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.90084.log: 88.2% 92.7% -- replaced with /var/log/ceph/cluster1-client.mirror.46262.log.gz 2024-04-23T22:00:13.231 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.90084.log.gz 2024-04-23T22:00:13.233 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.103715.log.gz 2024-04-23T22:00:13.235 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.101461.log 2024-04-23T22:00:13.235 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.81956.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.81956.log.gz 2024-04-23T22:00:13.235 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69135.log 2024-04-23T22:00:13.235 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.101461.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.101461.log.gz 2024-04-23T22:00:13.237 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97603.log 2024-04-23T22:00:13.237 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69135.log: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.69135.log.gz 2024-04-23T22:00:13.237 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.1.59058.log 2024-04-23T22:00:13.237 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.40488.log.gz 2024-04-23T22:00:13.239 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97603.log: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.53648.log.gz 2024-04-23T22:00:13.245 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56327.log 2024-04-23T22:00:13.248 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.1.59058.log: 90.4% -- replaced with /var/log/ceph/cluster2-client.mirror.97603.log.gz 2024-04-23T22:00:13.248 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.106511.log 2024-04-23T22:00:13.248 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.56327.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.86214.log 2024-04-23T22:00:13.248 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.106511.log: 77.7% -- replaced with /var/log/ceph/cluster2-client.mirror.106511.log.gz 2024-04-23T22:00:13.248 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65884.log 2024-04-23T22:00:13.250 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.86214.log: 91.7% -- replaced with /var/log/ceph/cluster2-client.mirror.86214.log.gz 2024-04-23T22:00:13.250 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69652.log 2024-04-23T22:00:13.252 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster1-client.mirror.65884.log: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.65884.log.gz 2024-04-23T22:00:13.252 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99544.log 2024-04-23T22:00:13.252 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.69652.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.69652.log.gz 2024-04-23T22:00:13.254 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99544.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.99544.log.gzgzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38490.log 2024-04-23T22:00:13.254 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:13.254 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.41846.log 2024-04-23T22:00:13.254 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.38490.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.36380.log 2024-04-23T22:00:13.256 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.41846.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.43640.log 2024-04-23T22:00:13.256 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.36380.log: 84.5% -- replaced with /var/log/ceph/cluster2-client.mirror.60232.log.gzgzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76896.log 2024-04-23T22:00:13.256 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:00:13.257 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.56327.log.gz 2024-04-23T22:00:13.263 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.43640.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.100208.log 2024-04-23T22:00:13.263 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.76896.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.76896.log.gz 2024-04-23T22:00:13.263 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70801.log 2024-04-23T22:00:13.263 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.100208.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.100208.log.gz 2024-04-23T22:00:13.263 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82582.log 2024-04-23T22:00:13.263 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.70801.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.70801.log.gz 2024-04-23T22:00:13.264 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.97961.log 2024-04-23T22:00:13.264 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.82582.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.82582.log.gz 2024-04-23T22:00:13.264 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64135.log 2024-04-23T22:00:13.264 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.97961.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.97961.log.gz 2024-04-23T22:00:13.264 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65083.log 2024-04-23T22:00:13.271 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.64135.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.38288.log 2024-04-23T22:00:13.271 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.65083.log: 1.3% -- replaced with /var/log/ceph/cluster2-client.mirror.65083.log.gz 2024-04-23T22:00:13.271 INFO:teuthology.orchestra.run.smithi150.stderr: 83.7% -- replaced with /var/log/ceph/cluster1-client.mirror.42540.log.gz 2024-04-23T22:00:13.274 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.43640.log.gz 2024-04-23T22:00:13.276 INFO:teuthology.orchestra.run.smithi150.stderr: 92.7% -- replaced with /var/log/ceph/cluster1-client.mirror.57169.log.gz 2024-04-23T22:00:13.277 INFO:teuthology.orchestra.run.smithi150.stderr: 84.6% -- replaced with /var/log/ceph/cluster2-client.mirror.60610.log.gz 2024-04-23T22:00:13.278 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.38490.log.gz 2024-04-23T22:00:13.278 INFO:teuthology.orchestra.run.smithi150.stderr: 92.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57235.log.gz 2024-04-23T22:00:13.279 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.47311.log.gz 2024-04-23T22:00:13.280 INFO:teuthology.orchestra.run.smithi150.stderr: 64.5% -- replaced with /var/log/ceph/cluster2-client.mirror.36380.log.gz 2024-04-23T22:00:13.283 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.46791.log.gz 2024-04-23T22:00:13.288 INFO:teuthology.orchestra.run.smithi150.stderr: 85.2% -- replaced with /var/log/ceph/cluster1-client.mirror.35627.log.gz 2024-04-23T22:00:13.295 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60017.log 2024-04-23T22:00:13.311 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.38288.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84270.log 2024-04-23T22:00:13.325 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.60017.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.88749.log 2024-04-23T22:00:13.325 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.84270.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.84270.log.gz 2024-04-23T22:00:13.336 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64342.log 2024-04-23T22:00:13.337 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.88749.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.88749.log.gz 2024-04-23T22:00:13.349 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74340.log 2024-04-23T22:00:13.349 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.64342.log: 1.3% -- replaced with /var/log/ceph/cluster2-client.mirror.64342.log.gz 2024-04-23T22:00:13.362 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.99337.log 2024-04-23T22:00:13.363 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.74340.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.74340.log.gz 2024-04-23T22:00:13.374 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.105351.log 2024-04-23T22:00:13.375 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.99337.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.99337.log.gz 2024-04-23T22:00:13.388 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% -- replaced with /var/log/ceph/cluster2-client.mirror.51443.log.gz 2024-04-23T22:00:13.389 INFO:teuthology.orchestra.run.smithi150.stderr: 93.2% -- replaced with /var/log/ceph/cluster2-client.mirror.0.49691.log.gz 2024-04-23T22:00:13.399 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67532.log 2024-04-23T22:00:13.401 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.105351.log: 91.3% -- replaced with /var/log/ceph/cluster2-client.mirror.105351.log.gz 2024-04-23T22:00:13.406 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.41846.log.gz 2024-04-23T22:00:13.409 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.40779.log 2024-04-23T22:00:13.409 INFO:teuthology.orchestra.run.smithi150.stderr: 94.1% -- replaced with /var/log/ceph/cluster1-client.mirror.49691.log.gz 2024-04-23T22:00:13.412 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.67532.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.67532.log.gz 2024-04-23T22:00:13.414 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.38288.log.gz 2024-04-23T22:00:13.416 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.55865.log.gz 2024-04-23T22:00:13.417 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.47858.log.gz 2024-04-23T22:00:13.418 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.107028.log 2024-04-23T22:00:13.419 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.40779.log: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.94651.log.gz 2024-04-23T22:00:13.431 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.61807.log.gz 2024-04-23T22:00:13.434 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.39234.log.gz 2024-04-23T22:00:13.434 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.39844.log 2024-04-23T22:00:13.434 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.admin.107028.log: 0.0%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.96652.log 2024-04-23T22:00:13.434 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.39844.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52038.log 2024-04-23T22:00:13.435 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.96652.log: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.96652.log.gz 2024-04-23T22:00:13.435 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.39135.log 2024-04-23T22:00:13.438 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.52038.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.36363.log 2024-04-23T22:00:13.438 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7%/var/log/ceph/cluster2-client.mirror.39135.log: -- replaced with /var/log/ceph/cluster2-client.mirror.68171.log.gz 2024-04-23T22:00:13.438 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.37980.log 2024-04-23T22:00:13.438 INFO:teuthology.orchestra.run.smithi150.stderr: -- replaced with /var/log/ceph/cluster2-client.admin.107028.log.gz 2024-04-23T22:00:13.438 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.36363.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82317.log 2024-04-23T22:00:13.479 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cluster2-client.mirror.37980.log: /var/log/ceph/cluster2-client.mirror.82317.log: 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.62847.log.gz 2024-04-23T22:00:13.488 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.86292.log.gz 2024-04-23T22:00:13.517 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.104562.log.gz 2024-04-23T22:00:13.523 INFO:teuthology.orchestra.run.smithi150.stderr: 92.5% -- replaced with /var/log/ceph/cluster2-client.mirror.1.59058.log.gz 2024-04-23T22:00:13.541 INFO:teuthology.orchestra.run.smithi150.stderr: 95.9% -- replaced with /var/log/ceph/cluster2-client.mirror.62669.log.gz 2024-04-23T22:00:13.546 INFO:teuthology.orchestra.run.smithi150.stderr: 92.9% -- replaced with /var/log/ceph/cluster2-client.mirror.45493.log.gz 2024-04-23T22:00:13.548 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.39844.log.gz 2024-04-23T22:00:13.563 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.64135.log.gz 2024-04-23T22:00:13.565 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% -- replaced with /var/log/ceph/cluster1-client.mirror.50785.log.gz 2024-04-23T22:00:13.573 INFO:teuthology.orchestra.run.smithi150.stderr: 92.8% -- replaced with /var/log/ceph/cluster2-client.mirror.57364.log.gz 2024-04-23T22:00:13.576 INFO:teuthology.orchestra.run.smithi150.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.55398.log.gz 2024-04-23T22:00:13.577 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.46985.log.gz 2024-04-23T22:00:13.580 INFO:teuthology.orchestra.run.smithi150.stderr: 92.7% -- replaced with /var/log/ceph/cluster2-client.mirror.45450.log.gz 2024-04-23T22:00:13.592 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.82317.log.gz 2024-04-23T22:00:13.636 INFO:teuthology.orchestra.run.smithi150.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.58092.log.gz 2024-04-23T22:00:13.676 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.52038.log.gz 2024-04-23T22:00:13.685 INFO:teuthology.orchestra.run.smithi150.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.54173.log.gz 2024-04-23T22:00:13.714 INFO:teuthology.orchestra.run.smithi150.stderr: 92.7% -- replaced with /var/log/ceph/cluster2-client.mirror.44506.log.gz 2024-04-23T22:00:13.717 INFO:teuthology.orchestra.run.smithi150.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.37980.log.gz 2024-04-23T22:00:13.718 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.39135.log.gz 2024-04-23T22:00:13.721 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.46826.log.gz 2024-04-23T22:00:13.748 INFO:teuthology.orchestra.run.smithi150.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.58049.log.gz 2024-04-23T22:00:13.809 INFO:teuthology.orchestra.run.smithi150.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.40779.log.gz 2024-04-23T22:00:13.810 INFO:teuthology.orchestra.run.smithi150.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.40876.log.gz 2024-04-23T22:00:13.812 INFO:teuthology.orchestra.run.smithi150.stderr: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.55595.log.gz 2024-04-23T22:00:13.813 INFO:teuthology.orchestra.run.smithi150.stderr: 84.4% -- replaced with /var/log/ceph/cluster2-client.mirror.59421.log.gz 2024-04-23T22:00:13.820 INFO:teuthology.orchestra.run.smithi150.stderr: 97.2% -- replaced with /var/log/ceph/cluster2-client.mirror.52423.log.gz 2024-04-23T22:00:13.832 INFO:teuthology.orchestra.run.smithi150.stderr: 4.7% -- replaced with /var/log/ceph/cluster2-client.mirror.36363.log.gz 2024-04-23T22:00:13.974 INFO:teuthology.orchestra.run.smithi150.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.57963.log.gz 2024-04-23T22:00:13.977 INFO:teuthology.orchestra.run.smithi150.stderr: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.1.53514.log.gz 2024-04-23T22:00:14.008 INFO:teuthology.orchestra.run.smithi150.stderr: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.56878.log.gz 2024-04-23T22:00:14.113 INFO:teuthology.orchestra.run.smithi150.stderr: 92.8% -- replaced with /var/log/ceph/cluster2.audit.log.gz 2024-04-23T22:00:14.173 INFO:teuthology.orchestra.run.smithi150.stderr: 94.2% -- replaced with /var/log/ceph/cluster1-client.mirror.1.60671.log.gz 2024-04-23T22:00:14.353 INFO:teuthology.orchestra.run.smithi150.stderr: 93.3% -- replaced with /var/log/ceph/cluster2-client.mirror.57986.log.gz 2024-04-23T22:00:14.353 INFO:teuthology.orchestra.run.smithi150.stderr: 92.7% -- replaced with /var/log/ceph/cluster1-client.mirror.44526.log.gz 2024-04-23T22:00:14.362 INFO:teuthology.orchestra.run.smithi150.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.55658.log.gz 2024-04-23T22:00:14.466 INFO:teuthology.orchestra.run.smithi150.stderr: 94.1% -- replaced with /var/log/ceph/cluster2-client.mirror.59484.log.gz 2024-04-23T22:00:14.560 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% -- replaced with /var/log/ceph/cluster2-client.mirror.49485.log.gz 2024-04-23T22:00:14.581 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% -- replaced with /var/log/ceph/cluster2-client.mirror.57642.log.gz 2024-04-23T22:00:14.663 INFO:teuthology.orchestra.run.smithi150.stderr: 92.1% -- replaced with /var/log/ceph/cluster2-client.mirror.51544.log.gz 2024-04-23T22:00:14.678 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% -- replaced with /var/log/ceph/cluster1-client.mirror.2.39387.log.gz 2024-04-23T22:00:14.759 INFO:teuthology.orchestra.run.smithi150.stderr: 92.6% -- replaced with /var/log/ceph/cluster2-client.mirror.3.51781.log.gz 2024-04-23T22:00:14.930 INFO:teuthology.orchestra.run.smithi150.stderr: 96.6% -- replaced with /var/log/ceph/cluster2-client.mirror.45535.log.gz 2024-04-23T22:00:15.218 INFO:teuthology.orchestra.run.smithi150.stderr: 91.2% -- replaced with /var/log/ceph/cluster2-client.mirror.0.35576.log.gz 2024-04-23T22:00:15.289 INFO:teuthology.orchestra.run.smithi150.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.50200.log.gz 2024-04-23T22:00:15.291 INFO:teuthology.orchestra.run.smithi148.stderr: 94.4% -- replaced with /var/log/ceph/cluster1-mon.a.log.gz 2024-04-23T22:00:15.350 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% -- replaced with /var/log/ceph/cluster1-client.mirror.49505.log.gz 2024-04-23T22:00:15.679 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% -- replaced with /var/log/ceph/cluster2-client.mirror.46285.log.gz 2024-04-23T22:00:15.679 INFO:teuthology.orchestra.run.smithi150.stderr: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.56425.log.gz 2024-04-23T22:00:15.679 INFO:teuthology.orchestra.run.smithi150.stderr: 93.2% -- replaced with /var/log/ceph/cluster2-client.mirror.57728.log.gz 2024-04-23T22:00:17.088 INFO:teuthology.orchestra.run.smithi150.stderr: 93.3% -- replaced with /var/log/ceph/cluster2-client.mirror.35548.log.gz 2024-04-23T22:00:17.569 INFO:teuthology.orchestra.run.smithi150.stderr: 95.9% -- replaced with /var/log/ceph/cluster2-client.mirror.60633.log.gz 2024-04-23T22:00:19.976 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.52527.log.gz 2024-04-23T22:00:20.697 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.50954.log.gz 2024-04-23T22:00:21.353 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.37142.log.gz 2024-04-23T22:00:23.016 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.39346.log.gz 2024-04-23T22:00:23.520 INFO:teuthology.orchestra.run.smithi150.stderr: 95.4% -- replaced with /var/log/ceph/cluster2-client.mirror.60485.log.gz 2024-04-23T22:00:25.720 INFO:teuthology.orchestra.run.smithi150.stderr: 95.3% -- replaced with /var/log/ceph/cluster2-client.mirror.60173.log.gz 2024-04-23T22:00:26.414 INFO:teuthology.orchestra.run.smithi150.stderr: 95.5% -- replaced with /var/log/ceph/cluster2-client.mirror.60558.log.gz 2024-04-23T22:00:27.933 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.45644.log.gz 2024-04-23T22:00:28.644 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.60375.log.gz 2024-04-23T22:00:29.424 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.51657.log.gz 2024-04-23T22:00:31.803 INFO:teuthology.orchestra.run.smithi150.stderr: 95.3% -- replaced with /var/log/ceph/cluster2-client.mirror.59446.log.gz 2024-04-23T22:00:32.989 INFO:teuthology.orchestra.run.smithi150.stderr: 95.4% -- replaced with /var/log/ceph/cluster2-client.mirror.59241.log.gz 2024-04-23T22:00:33.077 INFO:teuthology.orchestra.run.smithi150.stderr: 95.9% -- replaced with /var/log/ceph/cluster2-client.mirror.50873.log.gz 2024-04-23T22:00:33.223 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.60017.log.gz 2024-04-23T22:00:33.277 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.55703.log.gz 2024-04-23T22:00:34.394 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.59696.log.gz 2024-04-23T22:00:34.696 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.44813.log.gz 2024-04-23T22:00:34.812 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.55793.log.gz 2024-04-23T22:00:35.941 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.51740.log.gz 2024-04-23T22:00:36.793 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.59009.log.gz 2024-04-23T22:00:36.883 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.58669.log.gz 2024-04-23T22:00:37.357 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.49650.log.gz 2024-04-23T22:00:37.369 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.58935.log.gz 2024-04-23T22:00:37.420 INFO:teuthology.orchestra.run.smithi150.stderr: 95.9% -- replaced with /var/log/ceph/cluster2-client.mirror.52446.log.gz 2024-04-23T22:00:37.429 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.60322.log.gz 2024-04-23T22:00:37.675 INFO:teuthology.orchestra.run.smithi150.stderr: 95.5% -- replaced with /var/log/ceph/cluster2-client.mirror.58790.log.gz 2024-04-23T22:00:37.718 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.46442.log.gz 2024-04-23T22:00:37.773 INFO:teuthology.orchestra.run.smithi150.stderr: 95.9% -- replaced with /var/log/ceph/cluster2-client.mirror.58870.log.gz 2024-04-23T22:00:38.251 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.59844.log.gz 2024-04-23T22:00:38.856 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.45558.log.gz 2024-04-23T22:00:39.408 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.59610.log.gz 2024-04-23T22:00:40.139 INFO:teuthology.orchestra.run.smithi150.stderr: 90.5% -- replaced with /var/log/ceph/cluster2-mon.a.log.gz 2024-04-23T22:00:42.228 INFO:teuthology.orchestra.run.smithi150.stderr: 94.3% -- replaced with /var/log/ceph/cluster1-client.mirror.1.52568.log.gz 2024-04-23T22:00:42.476 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.59929.log.gz 2024-04-23T22:00:44.405 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.59170.log.gz 2024-04-23T22:00:44.570 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.39257.log.gz 2024-04-23T22:00:45.901 INFO:teuthology.orchestra.run.smithi150.stderr: 95.7% -- replaced with /var/log/ceph/cluster2-client.mirror.59777.log.gz 2024-04-23T22:00:46.272 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.44614.log.gz 2024-04-23T22:00:47.340 INFO:teuthology.orchestra.run.smithi150.stderr: 95.6% -- replaced with /var/log/ceph/cluster2-client.mirror.58731.log.gz 2024-04-23T22:00:48.628 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.37030.log.gz 2024-04-23T22:00:49.331 INFO:teuthology.orchestra.run.smithi150.stderr: 95.5% -- replaced with /var/log/ceph/cluster2-client.mirror.59303.log.gz 2024-04-23T22:00:49.554 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.60255.log.gz 2024-04-23T22:00:51.491 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.49550.log.gz 2024-04-23T22:00:52.240 INFO:teuthology.orchestra.run.smithi150.stderr: 95.8% -- replaced with /var/log/ceph/cluster2-client.mirror.46350.log.gz 2024-04-23T22:00:54.417 INFO:teuthology.orchestra.run.smithi150.stderr: 94.4% -- replaced with /var/log/ceph/cluster1-client.mirror.2.35508.log.gz 2024-04-23T22:00:58.397 INFO:teuthology.orchestra.run.smithi150.stderr: 94.2% -- replaced with /var/log/ceph/cluster1-client.mirror.1.59484.log.gz 2024-04-23T22:01:02.980 INFO:teuthology.orchestra.run.smithi150.stderr: 94.0% -- replaced with /var/log/ceph/cluster1-client.mirror.2.56767.log.gz 2024-04-23T22:01:08.586 INFO:teuthology.orchestra.run.smithi150.stderr: 94.1% -- replaced with /var/log/ceph/cluster1-client.mirror.0.51544.log.gz 2024-04-23T22:01:21.344 INFO:teuthology.orchestra.run.smithi150.stderr: 94.2% -- replaced with /var/log/ceph/cluster1-client.mirror.3.35548.log.gz 2024-04-23T22:01:24.028 INFO:teuthology.orchestra.run.smithi150.stderr: 94.3% -- replaced with /var/log/ceph/cluster1-client.mirror.1.35486.log.gz 2024-04-23T22:02:55.310 INFO:teuthology.orchestra.run.smithi148.stderr: 93.9% -- replaced with /var/log/ceph/cluster1-osd.2.log.gz 2024-04-23T22:02:59.151 INFO:teuthology.orchestra.run.smithi148.stderr: 93.9% -- replaced with /var/log/ceph/cluster1-osd.1.log.gz 2024-04-23T22:03:27.050 INFO:teuthology.orchestra.run.smithi148.stderr: 93.9% -- replaced with /var/log/ceph/cluster1-osd.0.log.gz 2024-04-23T22:03:27.052 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-04-23T22:03:27.052 INFO:teuthology.orchestra.run.smithi148.stderr:real 3m17.888s 2024-04-23T22:03:27.052 INFO:teuthology.orchestra.run.smithi148.stderr:user 3m41.650s 2024-04-23T22:03:27.053 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m9.499s 2024-04-23T22:07:42.575 INFO:teuthology.orchestra.run.smithi150.stderr: 94.2% -- replaced with /var/log/ceph/cluster2-osd.1.log.gz 2024-04-23T22:08:10.990 INFO:teuthology.orchestra.run.smithi150.stderr: 94.2% -- replaced with /var/log/ceph/cluster2-osd.2.log.gz 2024-04-23T22:08:56.794 INFO:teuthology.orchestra.run.smithi150.stderr: 94.1% -- replaced with /var/log/ceph/cluster2-osd.0.log.gz 2024-04-23T22:08:56.795 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-23T22:08:56.795 INFO:teuthology.orchestra.run.smithi150.stderr:real 8m47.598s 2024-04-23T22:08:56.795 INFO:teuthology.orchestra.run.smithi150.stderr:user 5m8.203s 2024-04-23T22:08:56.795 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m17.043s 2024-04-23T22:08:56.795 INFO:tasks.ceph:Archiving logs... 2024-04-23T22:08:56.796 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi/7670210/remote/smithi148/log 2024-04-23T22:08:56.797 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T22:09:28.866 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi/7670210/remote/smithi150/log 2024-04-23T22:09:28.867 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T22:10:13.709 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-23T22:10:13.723 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-23T22:10:13.724 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T22:10:13.727 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T22:10:13.853 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-23T22:10:13.866 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-23T22:10:13.866 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T22:10:13.869 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T22:10:13.883 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-04-23T22:10:13.886 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-04-23T22:10:13.915 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T22:10:13.915 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-04-23T22:10:13.915 INFO:teuthology.orchestra.run.smithi150.stdout:^- hv01.front.sepia.ceph.com 2 6 377 55 +1070us[+1070us] +/- 592ms 2024-04-23T22:10:13.915 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv02.front.sepia.ceph.com 3 6 377 56 -482us[ -463us] +/- 58ms 2024-04-23T22:10:13.915 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 121 +583us[ +604us] +/- 73ms 2024-04-23T22:10:13.915 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T22:10:13.921 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T22:10:13.921 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-04-23T22:10:13.921 INFO:teuthology.orchestra.run.smithi148.stdout:^- hv01.front.sepia.ceph.com 2 6 377 56 +1120us[+1138us] +/- 592ms 2024-04-23T22:10:13.921 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv02.front.sepia.ceph.com 3 6 377 55 -453us[ -435us] +/- 58ms 2024-04-23T22:10:13.921 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 +611us[ +611us] +/- 74ms 2024-04-23T22:10:13.921 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T22:10:13.922 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-23T22:10:13.936 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-23T22:10:13.937 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-23T22:10:13.950 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T22:10:14.043 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T22:10:14.130 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T22:10:14.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T22:10:14.159 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T22:10:14.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T22:10:14.188 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-23T22:10:14.200 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-23T22:10:14.212 INFO:teuthology.task.internal:Duration was 3506.652753 seconds 2024-04-23T22:10:14.212 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-23T22:10:14.232 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-23T22:10:14.232 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T22:10:14.235 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T22:10:14.281 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T22:10:14.298 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T22:10:14.552 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-23T22:10:14.552 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-04-23T22:10:14.552 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T22:10:14.572 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-04-23T22:10:14.572 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T22:10:14.593 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-23T22:10:14.593 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T22:10:14.615 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T22:10:14.661 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-23T22:10:14.661 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T22:10:14.680 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T22:10:14.853 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-23T22:10:14.872 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-23T22:10:14.872 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T22:10:14.913 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T22:10:14.959 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-23T22:10:14.972 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T22:10:14.974 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T22:10:15.005 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-04-23T22:10:15.012 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-04-23T22:10:15.030 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T22:10:15.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T22:10:15.085 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T22:10:15.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T22:10:15.099 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-23T22:10:15.120 INFO:teuthology.task.internal:Transferring archived files... 2024-04-23T22:10:15.121 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi/7670210/remote/smithi148 2024-04-23T22:10:15.121 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T22:10:15.158 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-23_20:16:13-rbd-main-distro-default-smithi/7670210/remote/smithi150 2024-04-23T22:10:15.158 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T22:10:15.190 INFO:teuthology.task.internal:Removing archive directory... 2024-04-23T22:10:15.190 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T22:10:15.194 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T22:10:15.239 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-23T22:10:15.254 INFO:teuthology.task.internal:Not uploading archives. 2024-04-23T22:10:15.254 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-23T22:10:15.267 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-23T22:10:15.268 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T22:10:15.270 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T22:10:15.282 INFO:teuthology.orchestra.run.smithi148.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 22:10 /home/ubuntu/cephtest 2024-04-23T22:10:15.293 INFO:teuthology.orchestra.run.smithi150.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 22:10 /home/ubuntu/cephtest 2024-04-23T22:10:15.294 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-23T22:10:15.307 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-23T22:10:15.335 INFO:teuthology.run:Summary data: description: rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_latest} workloads/rbd-mirror-journal-stress-workunit} duration: 3506.652753353119 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-23T22:10:15.336 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T22:10:15.430 INFO:teuthology.run:pass